Unspoken (2024)

electronic synthesisers, augmented reality interface

Unspoken is a mixed reality duet performance that explores the aesthetic possibilities of augmented reality technology as a medium for collaborative musical expression. Two performers – Yichen Wang and Sandy Ma – with their tangible musical systems are communicated of each other’s gestural and spatial musical intention through a collaborative augmented reality interface.

One performer manipulates a physical keyboard synthesiser, while the other uses physical drawing materials to control a digital synthesizer, finding a shared space for different practices to reach autonomous commonality. To juxtapose different forms of tactile soundmaking with a recognisable keyboard, this performance introduces a digital synthesiser system in PureData that applies the ML5 image classifier implemented in a web application to capture the performer drawing and manipulate the sound based on the classified object.

Unspoken was performed at the 2024 International Conference in New Interfaces for Musical Expression held in Utrecht, Netherlands.