Back To Portfolio

Platform: HTC Vive, 12-speaker surround (optional)
Duration: 15 Weeks
Language: C#, Unity ShaderLab, HLSL, SC Lang
Engines: Unity, SuperCollider
Team Size: 6
Roles: Graphics, Interaction, Systems Programmer

Music in Motion was a student-pitched project aiming to explore the relationship of physical music to sound creation in virtual reality. Over fifteen weeks, we explored around twenty prototypes in interactions, graphics, and sound to try to create an experience that would get the average person with no musical experience to feel like they have intuitive musical control in our environment.

As the lead graphics programmer, I was responsible for developing most of our visual effects and systems that would be strongly coupled with the interactions and sounds of our experience. Over the course of the project, I worked on GPU particle simulation, shader-based water simulation, mesh tessellation, and many externally controllable material and skybox shaders for our designers to adjust for the final product. The largest amount of work was put into the GPU particle simulation (based off of Keijiro Takahashi's Spray), where we prototyped several different visuals and interactions that utilized particles, such as an "aurora borealis" that can be influenced by the player and sound inputs, a stylistic hand render to make a lossy hand tracking method more appealing to the user, and an ethereal flowing line of particles that could be traced arbitrarily through space (which can be seen in our demo materials!).

Alongside my graphics development, I also contributed to the gameplay interactions and various game systems, including many positional and movement controls for our sound engine SuperCollider, efficient and uniform spline generation for the tracing paths, the build process to package our sound engine along with our game builds, and the DMX control system that we used to pair our DMX light setup with the lighting inside our experience.

Notable Experiences

Download NOTE: The game was originally designed for our custom 12-speaker setup using SuperCollider, which makes it somewhat fragile to port. If you have trouble running it but would still like to give it a try, shoot me an email!
Back To Portfolio