overview
The Synesthesia Drawing Machine is a program that brings the concept of synesthesia to life through an audio-visual experience. This drawing machine responds to audio input (either from a microphone or pre-uploaded songs) to create abstract art.
Role: coder
Program: Processing (Python)
design process
Inspired by the concept of synesthesia—a condition where stimulation of one sensory pathway leads to automatic, involuntary experiences in another, I implemented the functionality to register audio data from the microphone and pre-uploaded songs and to translate them into dynamic shapes and colors.
Based on the amplitude of the sound, I wanted the size of the shapes to be dynamically generated, so I used a function in Processing that analyzed the amplitude of the audio in real-time. Then I developed functions that used that data to set thresholds for the loudness, dividing it into three levels: loud, average, and soft. Each level corresponds to a different shape size, and I allowed users to also choose from various color and shape choices to create their evolving art.
As users switch between songs, turn on the microphone, and tweak the visual settings, the artwork becomes a personalized expression of the sound that fills the space. The machine essentially allows users to “see” music, much like experiencing synesthesia through sound.
sample art
April 2022
