These are some practical examples of original software and hardware tools developed as part the research underpinning my artistic productions and collaborations. This software devices are employed for sound synthesis, simulation of natural phenomena or human macvhine nteraction purposes.
Instruments
THE HIDDEN RESONANCES OF MOVING BODIES II (Piano & Marimba)
The physical gestures and movement qualities of the dancer are translated into music in real time by the mechanical piano, allowing her to play with the percussionist.
THE HIDDEN RESONANCES OF MOVING BODIES II (Interactive Dynamic Stochastic synthesis)
The dancer’s body activity controls the parameteres of a dynamic stochastic synthesis model. Each joint controls the parameteres of four waveforms that generated directly using stochastic funcions. This is instrument is inspired in Sergio Luque’s Stochastic Synthesis Concatenation extension to the Xenakis model.
PIANO & DANCER
The physical gestures and movement qualities of the dancer are translated into music in real time by the instrument. The dancer and the piano become two performers whose bodily movements are mutually interdependent. This interdependence reveals a close relationship between physical and musical gestures. Accordingly, the realisation of this insrument has been based on creative processes that merge choreographic and compositional methods. The interactive setting abolishes the necessity for a direct tactile manipulation of the acoustic instrument and thereby relaxes the traditional functional constrains of piano gestures. In addition, the project favours interaction techniques that take the expressive aspects of bodily movement into account and that integrate compositional algorithms as part of their mapping mechanisms.
ATAXIA
In this interactive instrument the dancer’s activity controls the accents of an isochronic pulse. Then her motion activity will affect also the height of the percussive sounds and the degree of order of this train of pulses. Thus the faster she moves the more the distances between pulses become unpredictable. The result is that the totally deterministic rhythm is transformed into a random one. Towards the end the dancers motion controls also the decay of the beats as well as their attack time. In this manner when the attack time is very long the pulses fuse into a mass. We have then operated a transformation from individuation to coalescence. All the sounds of this instrument are synthesized from scratch in supercollider language.
PRESSURE SENSITIVE SHOES
This project explores the creative possibilities of interactively controlling sound synthesis through pressure sensitive shoe inlays that can monitor minute body movements. It explores how small postural changes can be used to control music. From an artistic point of view, such an interactive relationship links the musical outcome of interaction to the proprioceptive awareness of a dancer and it exposes to an audience through the auditory modality a dancer’s minute movements that might be visually hidden. The project follows an approach that combines musical ideation, dance improvisation, interaction design, and engineering.
INTERACTIVE LASERS
Laser weareables that allow the user to create virtual architectures connected to sound synthesis. This device was designed for the scenic priduction The Marriage of Heaven and Hell, based on the homonymous work of William Balke. Technically, due to the highly focused characteristics of laser light, the lights extend the body of the dancers across stage. Therefore, the dancers bodies become actuators for shifting spatial geometries on stage. Conceptually, the strong contrast between darkness and brightness for lasers creates an interesting balance between sensory deprivation and sensory overload and thereby relates to the notion of the extended senses in William Blake’s text
INTERACTIVE LIGHT CONTROL
Interactive theater light control. Dancer trasnfers the energy of his body to the intensity of the lights. At the same time he is reading with his body William Blake’s romantic proverb One thought fills Inmensity, synchronizing it with his body movements and light changes in a multi-modal gesture.
INTERACTIVE DYNAMIC STOCHASTIC STEP SIZE MODULATOR
This instrument is an interactive version of a dynamic stochastic synthesizer. This method initially devised by Iannis Xenakis employs simulated brownian motion as a stochastic mechanisms to directly generate the sound pressure curve via the manipulation of individual digital samples. In this version the dancer modulates the step size of the random walk with her body motion. When the dancer’s activity is high the step size of the random walk increases resulting in very fast changes in the structure of the four voices that comprise the chord that are perceived as noise. When the dancer stops moving these four voices stop their frantic turmoil, stopping their pitch trajectory at different heights and resulting in new chords.
ATAXIA
Designed for the Motion Composer project in collaboration with Robert Wechsler and Infomus-Casa Paganini. In this interactive instrument the dancer’s activity controls the accents of an isochronic pulse. Then her motion activity will affect also the height of the percussive sounds and the degree of order of this train of pulses. Thus the faster she moves the more the distances between pulses become unpredictable. The result is that the totally deterministic rhythm is transformed into a random one. Towards the end the dancers motion controls also the decay of the beats as well as their attack time. In this manner when the attack time is very long the pulses fuse into a mass. We have then operated a transformation from individuation to coalescence. All the sounds of this instrument are synthesized from scratch in supercollider language.
VIRTUAL HAND EXTENSION
An artificial hand body structure that consists of six segments that are initially connected to the right hand of the dancer. In addition, a simple artificial neural network allows the dancer to control some of the structure’s shape properties. Throughout this scene, the number and position of the skeleton attachments changes. Each of these segments is composed of several springs, each of which is coupled to a synthesizer. The synthesizer is based on a synthetic model of a resonating string system written in Supercollidr language. Each synthesizer is composed of a variable number of integer multiples of a fundamental. These partials are perturbed by a brownian movement generator which adds a natural and organic quality to the sound. However, the perturbation of each partial does not deviate from its center frequency by more than 2%, which leads to a spectral fusion of the tone complex into a single pitched sound. This phenomena is described by Diana Deutsch in her article concerning grouping mechanisms in music .
INTERACTIVE SOUND SHREDDER
This instrument develops in an interactive real time fashion a sound transformation procedure invented by Trevor Wishart which he calls sound shredding. In this instrument the vocal sounds originally produced by the dancer are triggered with her sudden (thrust) movements and travel along an octophonic loudspeaker array. At each movement of the dancer this vocal multiplex is cut at a number of random positions in the file. The resultant segments are reassembled together in a new sound file which is cut again and again every time the dancer produces a violent movement. When this process is repeated a number of times the original sound is reduced to tiny fragments (shreds) that are perceived as a water texture. At this point the quality of the dancer’s movement changes into a floating quality and it is correlated with the amplitude of the resulting texture.
INTERACTIVE STOCHASTIC SWARM SEXTET
This interactive instrument combines artificial intelligence simulations of swarm behaviour with dynamic stochastic synthesis. The first 5 movements of the dancer give rise to 5 agents that will stop moving and freeze into a stochastic chord when the dancer moves, and dance frantically when the dancer stops. Accordingly this meta instrument is finally performed by an hybrid sextet comprised by 5 artificial participants and a natural one, the dancer.
BLINK MORPHER
In this interactive instrument the activity of the dancer’s eyes give rise to a sound that is successively transformed. In this case the dancer’s blinks metamorphose a crystal sound into a voice, which is actually her own voice.
NOWHERE
The dancer’s unrelated movements trigger the vocal-like synthetic sounds. The sounds are generated by an implementation of a dynamic stochastic synthesis model.