Kinect Hacking 103: Looking at Kinect IR Patterns.

So, I’m in the middle of reading every paper and patent related to Kinect, but in the meantime, I want to do something fun. Let’s look at the Kinect projector output.

The Kinect projector uses speckle to create this dot pattern. I will make an entire post about speckle, but in this case the IR laser is shining through a micro-patterned plastic lens. The resulting speckle pattern is “in focus” over a very large distance — tens of feet. For fun, I decided to place a book in the beam and look at just how it changed the pattern:

(large versions: frame 1; frame 2).

Here is the difference between those frames; the shift in the dots is immediately apparent.

Well, that was semi-informative. Interestingly, all the patents and papers related to the Kinect talk about creating the speckle dot pattern using a plain diffuser. While I’m going to demonstrate that in another post, this is NOT a random diffuser. In fact, the Kinect pattern repeats itself:

Here’s an animation of the different repeating elements laid on top of each other:

It’s not yet clear to me, however, if the repetition is actually being used by the Kinect, or if it is an artifact of micro-lithography/the diffuser printing process.

This entry was posted in ComputationalPhotography. Bookmark the permalink.

5 Responses to Kinect Hacking 103: Looking at Kinect IR Patterns.

  1. Joe Dunfee says:

    That infrared dot pattern looks very much like the technology used on the special paper used by the Livescribe Smartpen. The pen uses a camera to see a minute dot pattern printed on the paper, and from that can extract a location on the paper. This would allow the depth to be determined by using only a single camera. But I don’t actually know if this is the actual technology used on the Kinect.

  2. Daniel Huber says:

    The method that they use seems to be described in the patent here:
    http://www.wipo.int/pctdb/en/wo.jsp?WO=2007043036

    The depth is computed from the difference between the speckle pattern that is observed and a reference pattern at a known depth. The depth computation algorithm is a region-growing stereo method that first finds anchor points using normalized correlation and then grows the solution outward from these anchor points using the assumption that the depth does not change much within a region. There are a lot more details to the algorithm described in the patent, but that is the gist of it.

  3. Ismael says:
  4. Larry Cramer says:

    The speckle pattern is stil sharp a few meters away from the projector. If the camera is set to infinity, the dots on surfaces closer to the camera will become broader before they finally colesce. This can be converted to distance information by computing a single higher fourier coefficient for a small region. While shadow and dot displacements continue to grow even very close to the camera may be to much overlap of neighbouring dots seems to limit distance computation.

  5. Akos3D says:

    Hi Guys,

    This is a very interesting research about Kinect’s working method.
    I would be interested to talk to you regarding a possible use of the Kinect for 3D portrait photography.
    I already run a business for 3D Portrait photography, but I think there would be chance to make this business even bigger if Kinect could be used in a way I hope it could be used.
    Think something like the Kinect to be used while taking a photograph to get depth map.
    I know there are limitations, but I have some idea in mind which might overcome those issues.

    Thanks in advance,
    BR,
    Akos

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>