Multimedia Vivaldi Four Seasons
reactive performance

Processing, Max/msp, Touch OSC, iPad

in coolaboration with:
Yan Da, reactive designer
Daniel Abraham, conductor

March 7, 2011
Cultural Arts Center, Montgomery College
Silver Spring MD, USA


2012 How Interactive Awards Winner
 

Multimedia Vivaldi Four Seasons is a real-time reactive audio-visual performance. This collaboration between designers Yana Sakellion and Yan Da with Dan Abraham, a musical director of Bach Sinfonia, was performed at the Cultural Arts Center at Silver Spring, Maryland, USA. The concert featured responsive generative projections and particle typography of the original accompanying sonnets, possibly authored by Vivaldi Himself.


Our primary goal was to create a holistic experience between the visuals, typography and sound within the classical music context, and with consideration for the contemporary electronic audio-visual performances. Our secondary goal was to engage the demographic which otherwise may not be exposed to New and Interactive Media. We didn’t want to create a “music video”, neither did we want our work to become a mere subtitle backdrop to the musicians on the stage. Rather, we aimed to become the Visual Performers ourselves, and asked Dan Abraham to serve as our conductor.

The translations of the Sonnets appear at the exact times noted on the original score, and are composed of particle typography. The transitions timing of each sonnet line is pre-programmed, but the images that make individual words, as well as start/end queue points, are controlled live during the performance. What you see is a two-part system. The first (made with MaxMSP) captures the live sound and allows us to set its pitch and amplitude to different properties (position, speed, scale, color). The other (made with Processing) lets us map those properties to any image or text we are showing.


 

Before settling on a programmatic approach, we have explored various technology options, each with its capabilities and limitations. The imagery, which began as a traditional storyboard, evolved into a more flexible system of modular designs. We decided to create a visual instrument that could allow us to orchestrate and animate the foreground images (“notes”), background images (“sets”) and the texts (“phrases”) in real time. The “notes’” react directly to the live sound, while the “sets” and “phrases” are manually triggered.

The small pictures used to compose text required special treatment in order to be legible, yet expressive. The images follow the narrative of the sonnets, but do not reflect the text literally. Instead they set the mood and provide a playground for the imagination. This work will never appear exactly the same twice; much like the music itself would never be performed identically.

 

Credits:

Antonio Vivaldi, Le Quattro Stagioni(The Four Seasons)

Yan Da & Yana Sakellion, reactive media

The Bach Sinfonia
Daniel Abraham, conductor & artistic director
Ingrid Matthews, solo violin