Multimodal Looper / MSc Thesis
2023
–

The multimodal looper is an interactive tool connecting body movements to audiovisual forms for musical improvisation. Aimed for collaboration akin to a jam session, it is based on intuitive correlations that bind body movements to sounds and visual attributes. It extends the performative practice of the musical looper to create a live looping system with multimodal objects that are gesturally activated and visually represented.

Using a Kinect v1 depth camera for body tracking and Touchdesigner, the first prototype focused on creating intuitive cross-modal mappings and the essential modules of a gesture based audiovisual live-looping system. The main inspiration for the mappings was Pierre Schaeffer’s sonic objects: sound fragments defined by their multimodal features, such as iterative, impulsive and sustained.

Click on the images below to explore the multimodal objects.

Iterative

Sustained I

Impulsive

Sustained II

Sustained I