Soap and Milk

Research and Installation, 2016

Technical research for a high-speed and-high resolution body tracking system used for large-scale interactive installations.

We used an industrial infrared camera by Ximea to capture the movements of the users. The system is able to shoot 2k video in 170 frames per second, allowing the algorithm to never lose track of a moving body or its motion. In order to process this data, it uses a custom GPU based optical flow solution to analyze and compare 2 million pixels in under 5 milliseconds.

Within our research we found high frame rate and resolution to be more than just a boost of quality. Basic computer vision for interactive systems is often too slow to measure a user’s movement on a satisfactory level. The captured movement between two frames in time are essential for the algorithms outcome. At 170 fps the solution reconstructs more than the human eye is able to see. The resulting data reveals even the tiniest and fastest motion, allowing the user to feel their dynamics immediately.

The whole system runs on one computer, equipped with a strong processor and a Titan X. The camera image is uploaded to GPU, as soon as a new frame is detected. From this point on, all tasks ranging from image analysis to real-time rendering is performed on GPU only – to avoid bottlenecks of data processing and latency. The motion detection is done in openFrameworks and the rendering passes in vvvv. The communication between the applications is realized via Spout.

The tracking system is used in the »Soap and Milk« project by Waltz Binaire


#Computer Vision, #Infrared Light, #Shader


Concept and realization: Waltz Binaire

Art Direction: Christian Mio Loclair
Music: kling klang klong
Kamera: Julian Voltmann
Photo: Felix Albertin
Commisioned by: Identitätsstiftung

A more comprehensive description of the concept and more technical details can be found here: