Masterprojekt SoSe23

VamX-Virtual Audio Manipulation Experience

Acoustics are stimulating. They influence us in many different ways, create stress or comfort. Our stress level impacts our actions. But stress does not necessarily have to be negative. Depending on the task, stress can even be beneficial for the desired outcome. With VamX (Virtual Audio Manipulation Experience) we have developed a framework that supports the manipulation of sound in virtual environments and therefore also the manipulation of the stress level.

Goal

Our goal was to create a tool that helps Unity developers design applications that consider sound and its effect on the user. Either the desired sounds can be included in VamX or passed to the framework via FMOD.

Features

VamX

Vamx is a editor tool that allows multiple sounds instances to be grouped and controlled by a single component. The Editor component allows the user to import a sound file and manipulate various sound properties using different evaluation methods, such as custom graphs for complex sound manipulation. The various sound properties and their evaluation settings can be saved as presets and reused in other sound files. The Editor allows importing regular audio files via the Unity-Editor or FMOD-Events using the FMOD-Editor, that will automatically expose all custom automation/parameter sliders.

Each individual sound property expects a normalized float input value, that can be controlled by the user or by a script. Input values of ALL sound properties of a Editor component, can also be controlled and processed by a Editor Level.

The Manager component is a overview of all available editor instances in the scene. The editors can be directly manipulated from the Manager component. In addition to that, the Manager Level can control ALL editor instances and their respective audio properties, overriding the Editor Level. This value, similar to the Editor Level, can be controlled by the user individually or by a external script.

Heart Rate Sensor

In our demonstrations, we utilize the Manager Level by using a Bluetooth heart rate monitor device to measure stress levels through heart-rate variability (HRV). This value is then mapped to a normalized float value to control our entire sound system.

Games

There were three different interactive scenarios within VR to test and evaluate the effect of the framework, each with an unique approach to that task.

Tower of Hanoi

A low-stress task, which can be completed at its own pace.

Ball Maze

A stress level variable task in which you have to react on the virtual physics

Piano Tiles

Steadily increasing stress level due to this accelerating rhythm game. Can sound manipulation help us break the personal limit?

Team

  • Emircan Yüksel
  • Juri Wiechmann
  • Robin Jaspers
  • Ruslan Novikov
  • Tim Markmann

Betreuung

Alexander Kramer, Martin Steinicke

Links

https://showtime.f4.htw-berlin.de/ss23/master/m6-vamx

Download:

https://github.com/orgs/IMI-Project-Sound-Manipulation-in-VR/repositories

Skip to content