Master’s Thesis · Real-Time Audiovisual System · 2026
This project started with a question: What happens when sound and visuals do not simply react to an external input, but to each other?
For my master’s thesis, I developed a real-time audiovisual system in which sound influences a particle simulation, while information from the visuals is fed back into the sound. Every reaction becomes the starting point for the next one, allowing the system to develop differently each time it runs.
The main challenge was not simply to create change, but to find a balance between variation and control. I wanted the system to feel autonomous without becoming completely random or unstable.
The System.
The system is structured around three states: Growth, Saturation and Decay. Each state has its own visual behaviour and sound characteristics. Internal values determine when the system transitions from one state to the next.
During Growth, the particle system gradually becomes more active. Saturation increases its density and intensity, while Decay reduces and breaks down the existing structures before the cycle begins again.
Thresholds, parameter limits and a minimum duration for each state prevent the feedback loop from escalating too quickly. At the same time, the system has enough freedom to form different patterns and rhythms during every run.
I built the project in Unreal Engine 5.6 using Niagara, MetaSounds and Blueprints. Parameters such as particle spawn rate, attraction, noise and drag are connected to the sound and feedback logic. The visual output is then sent to TouchDesigner through Spout for further processing.
Research and Experiments.
The project is informed by cybernetics, particularly ideas around feedback, control and stability. Gestalt theory also played an important role in thinking about how individual visual and sonic elements can still be perceived as one coherent experience.
To understand what the feedback was actually adding to the system, I tested it in several stages. I began with a simple one-way connection in which the audio controlled the visuals. I then added the state structure, closed the feedback loop and finally compared different feedback strengths.
Each condition was tested repeatedly under the same setup. I looked at technical aspects such as stability, variation and state progression, but also at more subjective qualities like pattern formation, temporal development, apparent autonomy and audiovisual legibility.
Finding the Right Balance.
The experiments showed that more feedback did not automatically create a better result. Strong feedback produced more dramatic changes and made the system appear more autonomous, but it also pushed the particle simulation towards its limits more often.
A medium feedback strength created the most convincing balance. It allowed the system to develop visibly over time while remaining stable and readable. The bidirectional setup also made the relationship between sound and image feel more connected than in the earlier one-way versions.
The resulting prototype is the foundation for a larger audiovisual piece. In the next iteration, I want to make the state structure less rigid, allow the feedback to have more influence over what happens next and develop a richer audio layer. The aim is to turn the system into an audiovisual experience that feels alive and unpredictable, without losing its sense of coherence.
