Interactive Projection Mapping: 5 projects to discover
Introduction - interactive art and immersive experience projection mapping
Interactive art is a dynamic art form that reacts to its environment. The work produced thus creates a link between the environment, such as humans or nature, and the other elements that compose it. These elements can be a stage, a set, a sculpture, light, a computer system… In interactive installations, participants interact through sensors to produce a unique result generated live.
Example : Daniel Rozin – Mechanical Mirrors
Projection mapping, thanks to video projectors, makes it possible to adorn a space with light. It is therefore interesting for visual transcriptions of events occurring in the environment. These events can be for example the movement of a person, the action of a button, or a sound played by an instrument. In this article we offer a selection of interactive projection mapping projects in order to demonstrate the possibilities offered to create immersive experiences.
What is Projection Mapping?
Projection mapping, also known as video mapping or spatial augmented reality, is a cutting-edge technology that transforms ordinary surfaces into dynamic displays by projecting digital content onto them. This innovative technique has gained significant popularity across various fields, including marketing, entertainment, and education, due to its ability to create immersive and engaging experiences. By using video projectors, projection mapping can convey brand messaging, create unique stage backdrops, decorate walls and ceilings, and evoke powerful emotions. The versatility of this technology allows users to turn any surface into a canvas for stunning visual storytelling, making it a favorite tool for artists and marketers alike.
History and Development of Projection Mapping
The concept of projection mapping has a fascinating history that dates back to 1969, when Disney first introduced this technology at Disneyland. The initial use of projection mapping was in the Haunted Mansion ride, where faces were projected onto mannequin heads to create a spooky and lifelike effect. Since those early days, the technology has evolved dramatically, offering spectacular visuals that far outshine its humble beginnings. Today, contemporary projection mapping is a popular tool for creating immersive experiences, with applications that have expanded to various industries, including advertising, education, and entertainment. The advancements in projection mapping technology have made it possible to produce highly detailed and interactive displays, captivating audiences and transforming ordinary spaces into extraordinary environments.
Two interactive projection mapping projects by Jesse James Allen
Jesse James Allen began his career as a sound engineer in the entertainment industry in the early 90s, at the dawn of the digital age. He gradually became a media designer for theme parks and learned to master many tools, all of which he sees as essential pieces of a puzzle to deliver emotional experiences through storytelling. He has participated in the design of 53 video games and this activity has truly shaped his career as a creative. The influence of video games can be felt throughout some of his projection mapping projects where he sometimes uses video game controllers to control the projection.
Jesse sees huge potential for HeavyM in interactive experiences and exhibitions to tell more compelling and sensational stories. In fact, he works with painters and illustrators as bringing the physical tactile world into the equation makes the effect even more vivid, more organic. According to Jesse, “The mix of traditional art and digital creates a special kind of magic. I’ve been doing research and development with HeavyM for about a year and I feel like I’m just scratching the surface, touching on a whole universe of possibilities.”
His collaboration with Charlie Jicha resulted in a painting made with conductive ink. This ink is connected to a Touch Board, which converts the information into MIDI for HeavyM. Since HeavyM’s sequencer is MIDI controllable, Jesse linked parts of the painting to specific sequences. To effectively animate objects in projection mapping, it is essential to create masks that match the shapes of the surfaces being animated.
Based on this experience, Jesse James Allen envisioned and built the Seraph Angel, an 8-foot sculpture that was featured at Immerse in 2017, a huge arts festival hosted in Orlando by Creative City Project.
The Seraph combines projection mapping, motion capture and sound design for an immersive interactive experience. This is Jesse’s largest project to date with HeavyM, offering all viewers the opportunity to interact with the installation through their bodies and voices. Jesse harnessed the full power of HeavyM Live in this project.
The majority of the projected visuals are shaders, controllable by the visitor through motion recognition via Leap motion and touch interaction with the conductive ink. Finally, HeavyM’s sound analysis module was used to react to the sound of the visitor’s voice via a microphone.
Interactive projection mapping with motion tracking
Digital Essence produces interactive installations for corporate communication since 2014. The company relies on the collaboration between scenographers, computer graphics designers and engineers to design and implement interactive systems using video projection. For the Hacking de l’Hôtel de Ville de Paris 2020, an event dedicated to digital innovation, Digital Essence designed and produced an interactive projection mapping installation.
The main objective is to highlight a unique place, while proposing a digital animation enhancing the event. Thus, thanks to projection mapping, the steps of the main staircase react to the passage of the public by triggering video animations. The installation gives visitors an immersive and impactful experience they will remember. Multiple projectors were used to achieve high-quality projections that enrich viewer engagement.
Digital Essence used a ZED stereoscopic camera coupled with the Augmenta tracking software to detect the presence and position of people on the stairs. Edge blending was employed to enhance image quality and seamlessness across the projected surfaces.
This data is sent to a shader which is displayed and deformed with HeavyM in order to turn on/off the stairs when the person walks by. Finally, a “glitter rain” video is played as soon as someone is present in the detection field.
More information on this project: link to the blog article.
Personalize interactive displays with projection mapping
Another Digital Essence project: an immersive box offering a personalized experience based on choices made on a tablet. Following a series of questions, the visitor is immersed in a box with a projection around him. The experience can be customized according to the choice of colors, music and destinations. The animations and music vary according to the answers given, to ensure an impactful experience. Additionally, interactive games such as Virtual Rugby and Virtual Soccer can be integrated to enhance engagement through real-time interaction with the projected content.
For the technical part, the installation uses 2 Olga kits, 3 Full HD ultra-short-throw projectors, a computer and a tablet. Before entering the booth, visitors fill in a form on a web page displayed on the tablet. The answers are then entered into a database and processed by a program running on the computer broadcasting the show. Finally, this program controls the videos displayed by HeavyM thanks to OSC messages. The technology also allows for the transformation of walls into interactive displays, creating immersive experiences on customizable surfaces using various projection methods and interactive games.
Interactive projection mapping on stage
To end this selection of interactive projection mapping projects, we have chosen to show its application in a scenic context. For this performance linking dance and video projection, Diomedi Mattia has implemented a real-time image generation system, based on the dancers’ movements. The projection surface plays a crucial role in achieving dynamic visuals, transforming the stage into an immersive experience.
This interactive ballet was produced by Laserentertainment using Blacktrax technology and TouchDesigner. Blacktrax is a motion tracking technology that relies on infrared LEDs. The dancers are equipped with a small electronic circuit consisting of a battery and LEDs whose light is invisible to the human eye, but not to specific cameras. The positions of the dancers can then be known thanks to an image processing software and are sent to the TouchDesigner software. The unique features of this technology enhance the overall quality and effectiveness of the performance.
TouchDesigner allows you to program image generators through a system of nodes to be linked together.
The Future of Projection Mapping Projects
The future of projection mapping projects is incredibly promising, with interactivity poised to be the next major development. In the near future, we might see experiences akin to virtual reality (VR) using special glasses, allowing users to immerse themselves even further into the projected visuals. Technologies that enable viewers to influence the mapping in real-time through sound, touch, and sight are already making waves, enhancing the interactive experience. Wearable tech is also being integrated to provide even more personalized and engaging interactions. However, there are challenges that need to be addressed, such as the cost and production time involved in creating these sophisticated displays. The production process often requires collaboration among various departments and significant investment. Overcoming these obstacles will require continued innovation and investment, but the potential rewards in terms of creating truly immersive experiences are immense.
Conclusion
This selection of interactive projection mapping projects gives a glimpse of the applications and possibilities, showcasing the potential for creating an immersive experience. These are indeed very broad and will develop more and more in the years to come. Technology plays an essential role in interactive art because it is an integral part of the sensor systems that enable interaction. The accessibility of computer tools is therefore essential for the growth and multiplication of works in this discipline. HeavyM, with its ease of use and versatility, allows the realization of interactive projection mapping in a few clicks.
EN 
FR 
ES 
DE 
IT 
PT 
CN 
KR 
JA 
RU