Kinect on OSX Snow Leopard

Exhilarating Madness!

Microsoft's XB360 Kinect Sensor reached the market the other day to enrich games on the Microsoft Xbox 360 Console with biological motion detection and superb three-dimensional enhancements. Kinect has received good marks to date. The possibility of creating three-dimensional images with Kinect caught my eye yesterday in a post with the title "With Kinect Controller, Hackers Take Liberties" by Jenna Wortham published online in The New York Times Nov. 21, 2010. The potentials are intriguing. Watch Oliver Kreylos' demonstration below:

I decided to give this a try and compile the software driver needed to run the sensor on a MacBook running Snow Leopard (OSX 10.6.5). Sean Nicholls provides useful information in his post with the title "Compiling OpenKinect on a stubborn OSX 10.6.5 (Updated 21/Nov)", first published Nov. 13, 2010. Sean's instructions are straight forward, leading to a functional application that provides as with a simple start. Below I added a few annotations that may be helpful to the novice builder.

In order to obtain the source codes and compile them into binaries, we need to install the packages 'git' (git-1.7.3.2-intel-leopard.dmg) and 'cmake' (cmake-2.8.3-Darwin-universal.dmg), respectively. Both can be downloaded ready for installation from the links Sean provides in his post. Note that third party unix programs traditionally are installed into the directory /usr/local. The header files needed for compiling other packages are installed in the directory /usr/local/include. Compiled executable binaries will be installed in /usr/local/bin. Libraries will be found in /usr/local/lib, and files with information to be shared in /usr/local/share. This is the default map for the directory tree of all installs that conform to GNU conventions.

However, some software designers prefer to have their packages installed into their own directory tree in a separate subdirectory to /usr/local. This is the case with 'git'. We will find the package in /usr/local/git. If we wish to use 'git' commands in a terminal without providing the full path where they are located each time, we must add this path to our environmental PATH variable just so:

• export PATH="$PATH:/usr/local/git/bin

If we now type:

• echo $PATH

The terminal should print a colon-separated chain of bin directory paths ending with /usr/local/git/bin.

The next package to install is 'cmake'. The installer will prompt us to provide a directory where to install the executables; the default is /usr/bin. This is a directory for commands that the operating system uses. To avoid mixing system commands with others, I chose /usr/local/bin according to the above conventions. Again we add /usr/local/bin to our PATH variable as suggested above. Then we create a folder for the driver application, change to this directory in the terminal in which we previously set the PATH variable, and follow Sean's further instructions.

Note our additions to the PATH variable pertain only to this terminal window and are lost when we close it. To make them permanent, we need to edit the PATH variable listed in the .bashrc file in our home directory. But this is not strictly necessary for our purpose here.

Sean instructs us to download the sources for two packages that we must compile ourselves, using git: OpenKinect and LibUSB.  The first depends on the latter. Hence, LibUSB must be patched for the use with Openkinect, built and installed first, using the OSX generic make command. Note that the package will install into the appropriate subdirectories of /usr/local.

In the next step, we build OpenKinect, more precisely libfreenect, with cmake. The compiler needs to be pointed to the header files that come with LibUSB. This is accomplished with providing /usr/local/include/usblib-1.0/ on the appropriate line in the cmake configuration gui opened using the ccmake command as Sean illustrates. After cmake has configured the build successfully, we compile the package using the make command.

In my configuration, the build also needed to have the path to libjpeg specified, in order to successfully complete its last stage known as linking. The compiler complaint reads:

Linking C executable as3-server

ld: library not found for -lJPEG

In OSX, this library is located in: /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/Resources/. There must be ways to configure cmake with this path, probably by adding "-framework ApplicationServices -framework CoreFoundation" to the LDFLAGS variable. Proper configuration is crucial, if we wish to deploy our binaries to other systems. Because I was not planning to deploy the binaries, I decided to avoid re-configuration. Since the compiler always checks /usr/local/lib, I helped myself with providing a soft link in /usr/local/lib to the library's true location just so:

• sudo ln -s /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/Resources/libJPEG.dylib /usr/local/lib/libJPEG.dylib

With the repeat of the make step the build finished successfully and was ready to install.

Addenda

• I just received the Kinect and have been running a test for close to one hour without fail (11/29/10).

Kinect on the Mac

• The next step to more functionality entails hooking the kinect video stream up to an Actionscript3 flash server. The openkinect package provides help. Juan Carlos del Valle posted a video of the procedure to be followed with the title "Kinect now accessible from Flash" on the Freenect - Kinect projects site. The video moves along at a fast clip. I had to review the steps over and over again. Below I wrote out a step-by-step protocol of the procedure.
  
  Juan uses two Adobe Flash ActionScript files named test_depth.as and test_rgb.as which can be found in the libfreenect/actionscript directory. We need to download Adobe flex SDK_2.4 from the Adobe flex.org site to accomplish this. Once unzipped, we move the folder to our preferred location. The folder contains a bin-directory that is home to the mxmlc executable we need to use. Hence, it is helpful to add the path to this bin-directory to our PATH variable on the command line in the terminal window, as we added paths to bin-directories earlier. Then, we change into 'libfreenect/actionscript' directory using the cd command and type:
  
  
  • mxmlc test_depth.as
  
  With a little delay the terminal will respond:
  "Loading configuration file {your path}t/flex_sdk_4.1/frameworks/flex-config.xml
  {your path}/kinekt/libfreenect/actionscript/test_depth.as: Warning: This compilation unit did not have a factoryClass specified in Frame metadata to load the configured runtime shared libraries. To compile without runtime shared libraries either set the -static-link-runtime-shared-libraries option to true or remove the -runtime-shared-libraries option."
  
  Juan Carlos ignored the message. The command produces the file test_depth.swf in the same directory. We repeat the last step for test_rgb.as to generate test_rgb.swf.
  
  Next we need to start the Actionscript3 flash server. The subdirectory 'libfreenect/c/build/examples' containing the glview executable which generated the twin view in our earlier work is also home to the as3-server excecutable. Because I had not added this directory to my PATH variable, I changed into it and typed:
  
  
  • ./as3-server
  
  The terminal responded:
  Number of devices found: 1
  ### Wait depth client
  ### Wait rgb client
  ### Wait data client
  
  Opening test_rgb in my browser produced a live test image for color beginning with the comment:
  ### Got rgb client
  
  
  
  We kill the server with ctr C, and restart it with:
  
  • ./as3-server
  Then we opening test_depth.swf in our browser, producing a live image for depth beginning with the comment:
  ### Got depth client
  
  
  
  However, note that the red channel is not working. I am investigating (12/01/10).

• The red channel works with as3-server built using a current libfreenect clone obtained with:
  
  • git clone git://github.com/imekinox/openkinect.git
  In order to prompt cmake to include as3-server in the build, we must first change the target option for as3-server in CMakeLists.txt to "ON" (12/07/10).