Feiyang Zhou
Gallery

Hybrid Clips 6 of 12

VR physical-UI node editor

A modular synthesizer you patch with your hands: the UI is made of physical modules in VR.

OverviewOverview of a modular synthesizer built as physical nodes in VR.
Adaptive layoutLayout re-flowing as modules connect.
Connecting wiresConnecting two modules by hand.
Detach and reconnectDetaching and reconnecting modules.
Physical UIPhysical UI modules: knobs and sliders manipulated by hand.
Modules in BlenderThe physical UI modules modeled in Blender.

An experiment where the art and the system are the same object. Built in Unity for the Oculus Quest 2, it takes analog modular synthesizers as the model for a general node-based editor in VR.

I believe physical UIs are the most intuitive way to interact in VR, so every control is an object: knobs set an oscillator’s frequency, sliders its gain, and there are buttons and switches. The components were modeled with flame surfacing, a technique from vehicle design, then animated in Blender.

Each module can hold any number of input and output ports and physical controls, plus a decal to identify it. Ports are color-coded for input and output, wires show the direction of data and snap onto ports, and they can be attached and detached freely. The layout adapts as modules connect.

Tools

  • Unity XR
  • Oculus Quest 2
  • Blender

Do not press

This lets the cat loose on the page for 30 seconds. It will press things and knock the page about. You can stop it any time by grabbing the cat. Everything goes back afterwards.

Stop the cat?

Everything goes back where it was.