Sh4der Jockey is a program for creating visual performances or “VJ-ing” alongside a DJ. Unlike more contemporary tools like Resolume or VDMX, this tool does not rely on VJ loops or video files. Instead, it focuses on generating graphics via shaders. Some communities already exist which engage in shader live coding, or live coding in general such as Algorave and Demoscene. As participants of these communities ourselves, we wanted to create a shader live coding tool which suited our needs for creating performances in VRChat.

One such feature is the ability to combine multiple shaders into a custom rendering pipeline. We wanted to combine vertex shader effects with raymarching effects, or use an intermediate buffer for storing raytracing data among other things. There are other improvements compared to existing shader coding tools like dynamic MIDI bindings, stereo audio, or NDI video stream input. These features let us create effects for various DJs and we’ve used the tool in 9 performances so far.

Of course, nothing comes for free and developing this tool came with its own set of technical challenges. Some of these challenges include asynchronously compiling the shaders in order to reduce lost frames while a pipeline is being compiled, creating our own crates to satisfy features that we want, or making as much of the CPU multithreaded as possible in order to maintain frame rate.