Entries in Kinect (1)

Firefly + Kinect

Posted on July 7, 2011 by Registered CommenterAOP in , , , | Comments Off

Like many people, I've been anxiously awaiting the official release of the Microsoft SDK for the Kinect.  Now, that its officially out, I spent some time over the last two weeks working on a set of Kinect related components that I hope to include in the next release of Firefly (1.007).  The first component I tried to implement was the Skeleton Tracker... and I have to admit that the result are quite promising.  It's surprisingly fast and as long as you stay within the specified range of the sensor, the results are quite good.  Using this component I put together two very quick demo videos.

There has been a big push over the last decade to develop novel 3D technology for multimedia displays (whether its new ways for stereoscopic projection, refractive lens, etc.) One of the most successful implementations and inventive (in my opinion) was Johnny Chung Lee's reverse engineering of the Wii sensor bar.  Another recent example (and equally impressive) is this hack using the Kinect sensor and head tracking. 

The video above is my first attempt to create a real-time 3D display system within Grasshopper using Firefly's new Skeleton Tracker component and some simple camera manipulation. The Skeleton Tracker component outputs a list of points (click here for further explanation).  From there, I simply use the Horster Camera Control component (another 3rd party plugin for Grasshopper) to position the camera at the viewers head and the camera target at a point in space locating the Kinect sensor.  It really is that easy.  Turn on some real-time shadows and you've got a real-time 3D display.  It still needs some tweaking but it's pretty fun to play with.

This next demo shows how easy it is to turn gestural movements into physical actuation using an Arduino.  The setup is very simple.  My z-value of my right hand (basically the height of my hand) controls the brightness value (or Pulse Width Modulation - PWM) of the LED.  My left hand controls the servo.  When my hand is by my side, the servo goes to position 0 and if I raise my hand above my head the servo moves to position 180.  So simple.  Of course, this could be expanded to control all sorts of things... perhaps that is next.