Kinemetagraph, prototype

Kinemetagraph is a proposed interactive installation that invites the visitor to explore their physical relationship to the cinematic image.
This prototype was first prize winner at the 2015 NYC Media Lab Summit

ClientTyler Henry
Date2015
Category

Kinemetagraph reflects the bodily movement of the visitor in real time with a matching pose from the history of Hollywood cinema. To achieve this, it correlates live motion capture data using Kinect-based “skeleton tracking” to an open-source computer vision research dataset of 20,000 Hollywood film stills with included character pose metadata for each image.

This prototype was developed while a first year graduate student at Parsons MFA Design and Technology in New York.

Original slide presentation, with background and research and development notes:

Projector output capture from an early version:

 

 

Materials:
HD Projector, Kinect v1 depth sensing camera, Mac Mini. Images and pose metadata sourced from the Frames Labelled in Cinema dataset (by Carnegie Mellon University Robotics Lab) using MatLab.
Custom motion capture, search and visualization software built in C++/openFrameworks with OpenNI2 for skeleton tracking.