Mapping touch and replicating tactile sensations in technorituals
Introduction
TESSERATO is an installation where participants experience touches applied to a performer’s body through a haptic feedback suit. This suit is mapped to a structure where participants step into while watching meditative visuals, allowing them to feel pressure on the same points where the performer is touched.
Concept and Background Research
Tesserato, or tesseract, represents a cube that reveals all possible orientations within itself. This installation immerses you in one of the myriad alternate realities of your existence, offering a parallel journey for those confined to bed by illness or disability. Inspired by the Tupi myth of the universe’s creation – where a formless energy explored its own vastness and created the universe – Tesserato invites introspection and exploration of inner worlds.
Created as a PhD research prototype, Tesserato aims to help people with disabilities experience the outside world through haptics. As someone with a chronic illness, I developed methods of imaginative meditation during moments of physical limitations. Tesserato allows participants to explore unfolding visuals of tesseracts – alternative realities – while engaging in tactile sensations.
The performer’s suit triggers motors in the structure, simulating touches felt by the performer. The patterns were made using Adobe Illustrator, with the AI feature of translating textual prompts into images to be sent to a digital sewing machine. In this case, the prompt use was the myth itself, ran through the algorithm until it produced similar visual references to the indigenous communities and that, through the lens of tech, shared the story. Essentially, these patterns offer a haptic translation of indigenous practices in the post-digital world.
Image 1: Woman from the Jurana community drawing patterns on a paddle.
Image 2: Jurana pattern for body painting.
Image 3: Detail of the basket weave, made from arumã splints.
Image 4: Family of the Wayana community.
Image 5: Pattern sewn into part of the suit.
Image 6: Back of the pattern with conductive thread.
Image 7: Performer wearing the suit.
Technical Implementation
The technical aspects of the piece fall into three categories: input, output, and visuals.
The visuals, created in TouchDesigner, feature a loop of infinitely unfolding cubes, referencing the mathematical term tesseract and providing a meditative experience.
Image 8: visuals created in TouchDesigner, which were watched by the audience when stepping into the structure.
The input consists of a suit worn by the performer with patches sewn with conductive thread. Each pattern connects to an MPR121 touch sensor, which is mapped to the corresponding output.
Image 9: MPR121 sensor connected to the PCB and Arduino Mega.
The output is a wooden structure with five stepper motors, each linked to a thread in the suit. Motors are positioned where corresponding body parts are mapped. Motor 1, in the center of the structure, connects to the thread on the performer’s chest. Touching patch 1 on the suit triggers motor 1 to apply pressure to the participant’s chest.
Image 10: Back of the structure with the wiring of all motors and computer running.
Motors 2 and 3 are placed on the participants’ upper arm, and are mapped to the performer’s Same with motors 4 and 5, but to the lower arms. These pull rather than apply pressure. Participants wear sleeves with strings pulled by the motors when triggered by patches. These sleeves, with fabric outside and soft silicone inside, simulate skin texture and human touch.
Image 11: close up of the skin texture on the silicone in the inside of the sleeves.
Image 12: string system of the sleeves seen from inside and outside.
Image 13: close up of the sewing of the sleeves, viewed from the inside.
Reflection and Future Development
I’m currently applying to PhD programs to expand my research. I’m interested in diving into how we can combine more inputs and outputs to make touch mapping more precise. I’m also excited to explore how to simulate a wider range of sensations, not just touch and pressure – like a stroke, a poke, or even pain.
I had planned on making this interaction wireless using OSC, which would allow for mobility. Now, I plan to do this with a more portable design, without sacrificing accessibility. Plus, I want to experiment with recording specific touches so they can be replayed later. For instance, could I feel my mother’s hug from a distance? Could I record and revisit that hug years down the line? These are the kinds of possibilities I’m hoping to explore.
References
Damasio, Antonio. The Feeling of What Happens: Body and Emotion in the Making of Consciousness. Harcourt, 1999.
Nicolelis, Miguel. Beyond Boundaries: The New Neuroscience of the Brain and the Mind. Harvard University Press, 2011.
Sacks, Oliver. The Man Who Mistook His Wife for a Hat and Other Clinical Tales. Summit Books, 1985.
Castaneda, Carlos. The Teachings of Don Juan: A Yaqui Way of Knowledge. University of California Press, 1968.
Fried, L.MPR121 12-Channel Capacitive Touch Sensor Library, Adafruit Industries. Available at: https://www.adafruit.com/products/
Abramović, Marina. The Artist Is Present. Museum of Modern Art, 2010.
Rotermund, S., Associação Ita Wegman, Curitiba, 2016.
Bress, Eric and Mackye Gruber, J. The Butterfly Effect. New Regency Pictures, 2004.
Jonze, Spike. Being John Malkovich. Universal Pictures, 1999.
Tender World, Generate shapes from an image with TouchDesigner, YouTube, 2024. Available at: https://youtu.be/98xNOgU1zeI?si=rjNQzYR4cSAmql-D
supermarket sallad, Deconstructing Sculptures, YouTube, 2024. Available at: https://youtu.be/ruHVM5KZjB8?si=4g9s3zWoZPND1KC-
elekktronaut, Moving Through Shapes, YouTube, 2019. Available at: https://youtu.be/uKrA3nZSK4k?si=M0bWZup90Sz-DG4i
Sheenagh, Pietrobruno. (2009). Cultural Research and Intangible Heritage. Culture Unbound : Journal of Current Cultural Research. 1. 10.3384/cu.2000.1525.09113227.
Abbott, Edwin A. Flatland: A Romance of Many Dimensions. Seeley & Co., 1884.
Tingewly, Jean. Omnipresence in Digital Art. Art Forum, 2022.
Wenders, Wim. Until the End of the World. Warner Bros., 1991.
ChatGPT was used for troubleshooting, but no code written by it was used.