Earlier this month Adafruit started a contest for the first person to hack the Kinect, Microsoft’s latest gadget for the XBOX 360. The contest was won by Spanish hacker Héctor Martín Cantero who published his proof of concept only 3 hours after the European launch (last week).
So what exactly is the Kinect? Is it similar to the PS2 eyetoy?
Actually it is much more advanced than the PS2 eyetoy. Unlike the eyetoy, the Kinect contains 2 cameras. One RGB camera which is used for ‘normal’ images and a depth camera is used to figure out the position of objects in its view. A nice explaination (and an overview of the components) can be found at ifixit.
Because Adafruit required the contest winner to open source his source code, this means that others can now enjoy hacking their Kinect as well! Currently the code is available at github and renamed to libfreenect (irc: #openkinect @ freenode): https://github.com/OpenKinect/libfreenect/
Getting the code to compile might be a bit tricky and it involves CMake to create the project files. Running it on Linux is very trivial (just make sure you’ve installed all dependencies), on Windows and Max OS X there are some extra steps involved to compile the library and demo application.
For the windows version you will need libusb, glut and pthreads. Also, don’t forget to select the win32 branch when you do a checkout from git. After creating the visual studio project files you will need to manually fix the path to the include and lib directories (The current CMake file is broken).
If you can’t be bothered, I have compiled a windows binary (vs2k8): Kinect-VC9-Release.zip
Before running the binary, make sure you’ve installed the drivers from github (First the XBOX motor, then camera and audio). To control the motor in the Kinect, you can use this code: NUI_Motor.cpp
My experiments
Last Friday I actually bought myself a Kinect. In the video below you can see me running the Kinect on my machine. For now it is just retrieving both camera streams and putting it on display.
Basically the library (libfreenect) is providing you with two images through a callback. One depth image and one RGB image. The depth image actually maps a depth to a certain color. In this video for example, the color red/white means something is really close to the camera and green/blue is further away.
Hopefully I will have some nice apps later this month :).
Kinect hacks by others
3D mapping by Oliver Kreylos
ofxKinect 3D draw by Memo
Multitouch hack by Florian Echtler
Kinect Point Cloud by cc laan
Contest
P.s.: Before I forget, Matt Cutts (Google) started another contest. Check it out at: http://www.mattcutts.com/blog/open-kinect-contest/
One response
What’s most fascinating about this is how far the product has already come! From even this post there’s be a tremendous amount of movement in the hacking community. Just take a look at this latest hack of a Quadrotor flight robot/machine that has autonomous Kinect navigation. Simply remarkable the way that it detects obstacles and how the computer can automatically adjust its flight path. Sites like OpenNI, OpenKinect are really helping to facilitate this community.