Create an interactive projection with the Open Sound Control
Impressive Hakanaï performance by Adrien M & Claire B
Interactive art
Interactive art is a dynamic art form that reacts to its audience or environment. The works are generally presented in the form of installations in which the public can take part. They can also be experienced through performances during which one or more artists carry out actions.
Interactive art appeared in the late 1950s following the desire of artists to exhibit in places less exclusive than museums or galleries. Streets and stores became spaces of expression in order to propose more participative and inclusive experiences. The participants are no longer spectators and their role can then resemble that of an actor.
The earliest examples of interactive art were sculptures that could be touched and played with. For example, Niki de Saint Phalle produced Golem, a monumental sculpture that has a slide for children to play with. Another example, in 1971 Gordon Matta-Clark cooked a pig under the Brooklyn Bridge and served 500 sandwiches as part of the performance.
Some corporate events implement these processes, for example in product launches, to engage their audiences to leave them with a memorable souvenir or offer a personalized experience.
From a technical point of view, interactive performances or installations are increasingly using computers and interactive projection. Sensors are used in combination with computers so that events in the real world can be interpreted by programs and then transcribed, for example in sounds or images.
Open Sound Control (OSC) for interactive projection
Interactive art is based on the dialogue between several entities. Usually these entities are not natively designed to dialogue with each other. For example, a music software and a video software installed on the same computer, a Mac and a PC on the same network, or a human being and a machine in the same room.
Standards exist to facilitate exchanges between different software and hardware. We talk about network protocols, such as UDP or TCP, to designate the rules that govern the interactions between the different entities on the same network. Among the standards used, there is also the Open Sound Control which is a transmission format based on UDP or TCP.
Protocols can be compared to rules in a discussion, such as not cutting each other off. Formats can be compared to the grammar of a language, for example starting a sentence with the subject. If everyone follows the same standards, everyone understands each other!
Open Sound Control is a format inspired by MIDI (Musical Instrument Digital interface) used in communications between musical instruments since the 1980s. It takes up this idea of connectivity between instruments and software, through the exchange of messages, but with significant improvements. The OSC can use a network connection such as the Internet or a local network, it can transport more information and it offers an address-based system, similar to file paths.
Originally designed for music applications, OSC has now become a standard used by hundreds of software applications. It is no longer limited to audio data and is capable of carrying images or information for motion tracking for interactive projections as an example.
Some examples of interactive art softwares compatible with Open Sound Control: PureData, Vézer, IanniX, Chataigne, Max/MSP, OpenFrameworks, Processing, HeavyM,TouchDesigner…
Depending on their use, these softwares allow you to control with external data an interactive projection generated with HeavyM for example. Two programs that caught our attention are Vezer for the Mac and Châtaigne for the PC. With these tools you can control HeavyM with oscillators or variation curves that you draw beforehand.
Regarding the interactions between humans and computers, we speak of HMI for Human-Machine Interface. HMIs are the visible surfaces of software and they allow users to manipulate the functionalities. In fact, you are currently using the HMI of your Internet browser, which allows you to read these lines. The hidden part of the browser allows you to exchange information with other computers and servers on the Internet.
Hexler develops TouchOSC, a tool for building HMIs to send OSC messages. With TouchOSC, you can create applications, usable on tablets and smartphones, to control OSC compatible softwares. Discover here the technical information to connect TouchOSC to HeavyM.
To make using TouchOSC with HeavyM easier for you, we offer pre-designed interfaces available for download here. These interfaces incorporate the main features of the software and you simply need to install them on your tablet to remotely control HeavyM.
Finally, in addition to software, it is possible to use devices capable of driving software in OSC. This is the case, for example, with the Stream Deck created by Elgato. This box allows you to simply control the functionalities of an interactive projection software by assigning a key. The main advantage of this system is that the keys are miniature screens and you can therefore display an icon on them. It’s very easy to get around!
Example: how Data Flaw created an interactive projection using Open Sound Control?
Romain Astouric, aka Data Flaw is passionate about the digital revolution we are living in. It affects all artistic fields, with the production of more and more works using technology in music, visual arts and especially interactive art. Technology is democratized and is now just a click away. Artists now have access to affordable and easy-to-use tools for interactive projections.
Data Flaw, as a creative technologist, teaches coding and the use of technology for creative purposes at ESME Sudria, an engineering school in Paris. He finds inspiration in the work of artists such as 1024 Architecture, Joanie Lemercier, or the Scale collective. In his artistic work, Data Flaw regularly uses projection mapping and welcomes its opening to a wider audience thanks to tools like HeavyM.
For his project at the Makerlab of Paris Interdisciplinary Research Center, Data Flaw integrated a notion of interactivity to integrate the viewer into the work. The desired approach is both aesthetic and playful. The public becomes an actor of the installation and becomes aware of the consequences of its actions on the result displayed as an interactive projection.
The installation is a projection surface made with digital printing and cutting tools, on which visual animations generated by HeavyM software are projected. The artist used a popular accessory, a Playstation 5 controller, connected to HeavyM in OSC, through a program created with Processing.
The joystick can trigger various sets of effects and sequences. For example, it can control the color of the shapes and their colors using the joysticks. Anyone can easily create beautiful live color combinations. The color palette and numerous gradients are inspired by a particular video game: Hyper Light Drifter.
Unlike a tangible object, computer code is never finished. There are always features to improve or new bugs to fix. This is what Data Flaw is particularly fond of. For his projects, he always needs to fine-tune, with a lot of back and forth, testing and trials. HeavyM is an indispensable part of his creative process, both in the prototyping phase and in the final adjustments.
With HeavyM, you can edit the interactive projection, modify or add shapes with just a few clicks without having to start over or go through a rendering stage. It is a flexible but powerful tool, without unnecessary complexity.
The democratization of technology now allows a multitude of artists to express their creative talents. More and more interactive installations are being created and the public is in demand for this type of experience, both aesthetic and playful. To get inspired, you can find examples of creations on Design Boom or Trend Hunter.
To be continued
Learn more about projectors
We have created a detailed guide to choosing the right projector for your needs. Discover our tips and mistakes to avoid in this guide that explains the different features step by step.
Explore the HeavyM software
HeavyM is a video mapping software that is capable of generating visual effects and projecting them on volumes, decorations or architectural elements. HeavyM is Mac and Windows compatible.
EN 
FR 
ES 
DE 
IT 
PT 
CN 
KR 
JA 
RU