Composing melody with Xbox Kinect

Ken Stewart is a graduate student in the department of music. He is a music composer. By dancing in front of his Xbox Kinect to compose music, he changed the entire meaning of “Dance Your Ph.D”.

He used specialized software and a camera to dictate sounds. He presented this at “Visualization Friday Forum” held on January 27. This enabled the audience to have a glimpse of behind-the-scenes and the expressive way he used to compose music.

Kinect comes with a motion detection capability which is used in video games. By making use of this technology, Stewart started creating sounds with his gestures. He could also control the pitch, loudness and intensity of scores. The technology he used could record 15 movements of his body that were made by head, shoulders, neck, feet and knees.

He used a cache of sounds and then controlled the composition with his choreography. The computer could calculate the distance between the camera and his body and the angle at which his hands were moving. All such recordings were converted into music notes.

Stewart said that this way he could use his actions and ears to compose music. He also said that there were some mismatches between the moves and notes recorded. Stewart will be performing it again at Berkeley, Belfast, Grand Rapidly and Calif.