MoshBox

This is the operating manual for MoshBox. It is a live datamosh instrument: one video input, one moshed output, a small grammar of packet operations, and an eight-bank frame tape.

Its behaviour is informed by Interstream v2.0 by Jonas Bers / Signal Culture, but the implementation is new.

Signal flow

Camera or clip ──► encoder ──► mosh grammar ──► decoder ──► HDMI
                                      ▲                │
                                      └── frame tape ◄─┘

The encoder and decoder form one continuous session. Sources are normalised to the scene size before encoding, so switching between them does not reset the codec. A source change is an event the grammar sees: the SourceSwitch op makes it a clean cut or a smear that drags the previous reference into the new picture.

The frame tape sits alongside the output. It stores decoded pictures for time-domain repeat and, in Mosh mode, can feed them back through the encoder.

Setting up

You need:

• One USB UVC camera or capture device (the main source)
• An HDMI display, mixer or capture card
• A MIDI controller for hands-on control; the default map is for a Korg nanoKONTROL2
• A power supply suitable for your Raspberry Pi device (5V / 2.5 A for Pi 3 or Zero 2, 5.1 V / 3.0 A for Pi 4)

Connect the camera, HDMI and MIDI, then power up. If no camera is attached, MoshBox plays the Autumn Sun over the river Spree, so the engine still has something to work with.

The output resolution is baked into the image. The standard Raspberry Pi 3 build is 640×480, a 4:3 SD image for analogue chains. The Pi 4 build is 1280×720, a 16:9 HD Ready image. Capture, scene and display share one resolution, so the picture is not rescaled on the hot path.

Sources

The camera is the shipping source and hot-swaps: plug a capture device in after boot and MoshBox opens it. The clip is the fallback. It is the first video file by name in the data folder, looped, and it is the drop point for footage without a rebuild. There is no playlist and no USB watching.

• s switches to the next source. On the nanoKONTROL2, the TRACK buttons do this.
• c selects the camera.
• C cycles capture devices.
• l jumps straight to the clip.
• The SOURCE SWITCH pot sets what happens at a source change: 0 is smear, 1 is clean. The default is clean.

Sources are cover-cropped to the scene's shape before encoding, so a camera and a clip of different shapes arrive at the encoder as the same format.

Switching sources mid-mosh is the point of the single-stream design. With the SOURCE SWITCH pot toward smear, the new source inherits the old reference and the change melts; with clean, a keyframe is forced and the cut is hard.

The mosh grammar

MoshBox treats the encoded stream as a list of packets and runs a program of ops over it. A program is an ordered list of ops with parameters, a seed and a tempo.

The default program is SourceSwitch + DropKey + RepeatDelta. DropKey withholds keyframes so the decoder predicts from a stale reference; RepeatDelta replays delta packets so the motion keeps accumulating. That is the continuous mosh. Press mosh (space, or SOLO 1 on the nanoKONTROL2) to engage it.

The program runs a small state machine. Idle is a clean pass-through, Active is the mosh, and Recovering lets a clean reference through while repeat and damage ops are quiesced. That is why Normalize can snap the picture back to a clean reference.

Presets can carry other combinations of these ops. The default program is not the only program.

Mosh modes

MoshBox has a continuous mosh and two optional layers:

• Continuous — the default program. A steady pour with no clock.
• Rhythm — adds Pulse, which opens and closes the mosh on the beat. The PULSE DIV and PULSE HOLD faders shape the gate. Toggle it with p or SOLO 3.
• Motion — the source's own motion drives RepeatDelta.count, so the mosh stretches harder when the picture moves. Toggle it with m or SOLO 4; the MOTION AMOUNT and MOTION MAX pots set the depth and ceiling.

The layers can be combined. Continuous is the foundation; rhythm and motion are layers on top. Rhythm uses the clock: the tempo can come from the internal BPM, MIDI clock, or tap tempo. Presets capture the tempo.

Frame tape

The grammar's repeat ops work in the packet domain: they re-apply motion, so the picture stretches. The frame tape is the other kind of repeat. It stores decoded pictures, so it can play a moment again exactly — or feed that moment back through the current program.

The tape has eight banks. Each bank can hold a loop of 1 to 120 frames (4 seconds at 30 fps). It runs at 30 fps, independent of the render rate. A bank is captured from the live history; the default loop is the most recent 4 seconds.

Banks

• Press an empty bank (1–8, or REC 1–8) to capture the last loop and start playing it.
• Press a filled bank to recall it.
• Press the active bank again to toggle between the loop and live passthrough.
• x captures into the active bank, overwriting it.
• k clears the active bank.
• j freezes or resumes the playhead.
• y toggles loop / live.
• i cycles the loop mode: forward, ping-pong, reverse.
• , and . change the loop length.
• < and > move the loop window.
• The MARKER RIGHT button restarts the active loop.

Wet, dry, clean and mosh

• Wet is the decoded output — the show, mosh included.
• Dry is the encoder input — the source before the mosh.
• Clean displays the stored frame directly, with no generation loss.
• Mosh feeds the stored frame back through the encoder, so the current program chews the memory.

u switches between wet and dry. They are captured on the same tick, so the switch is a frame-exact A/B at the same playhead. n switches between clean and mosh playback. In Mosh mode the dry track is often the more interesting one: it re-performs an old moment under the current settings rather than replaying the old result.

The loop window

The loop window is a slice inside the captured bank. The loop length sets its size, and the position moves its centre: 0 is the earliest window, 1 is the latest. The window is per bank, so switching away and back restores where you left it. A fresh capture starts on the most recent window; shortening keeps the window's end fixed, so a one-frame loop freezes the newest frame.

Recording

Clean mode keeps recording the live pipeline, so you can capture a new moment while watching an older loop. Mosh mode pauses recording, because the encoder input is the tape itself and recording would capture the feedback path.

Banks share the frame pool rather than copying it, so eight overlapping loops are cheap. If the pool fills, the oldest captured bank is evicted.

Presets capture the tape's transport — loop length, loop mode, track and Clean/Mosh — but never the frames. Bank contents are performance material, not configuration.

Order modes

The order layer reorders packets inside the window. It adds about eight frames of video latency; control latency is unaffected.

Use o or MUTE 6 to cycle the modes. MUTE 1–5 select a mode directly.

Presets

A preset captures the whole program: the ops in order, their parameters, the seed, the tempo, the encoder settings and the tape transport. It never captures the tape's frames. Recalling a preset and feeding the same input reproduces the same output.

• REW and FF recall presets, and the MARKER SET button saves.

Presets live in the app's data folder.

Encoder

Codec, bitrate and GOP are live controls.

• CC36 cycles the codec: mpeg4 → mpeg2video → h263p → libx264 → libvpx.
• CC17 halves and doubles the bitrate. A lower bitrate is coarser and blooms more; a higher bitrate is finer.
• CC21 steps the GOP up and down by five frames.

A codec change rebuilds the decoder and forces a keyframe, so the stream recovers immediately. An invalid setting rolls back to the previous working encoder.

The default codec is mpeg4. Change codecs in discrete steps; sweeping through them is not useful.

MIDI — nanoKONTROL2 default map

MoshBox opens a MIDI port by name. The default match is nanokontrol; set MOSHBOX_MIDI to use a differently named or virtual port.

The default map splits by function: the channel strips play, the SOLO row is the mosh and tape layer, the MUTE row is the order layer, the REC row is the eight tape banks, and the master/transport section navigates presets and sources.

Faders

Pots

Pot 8 (CC 23) is unbound.

Buttons

MUTE 7 and MUTE 8 (CC 54/55) are unbound. The default map leaves three of the 51 controls empty on purpose.

The buttons are momentary CCs; each press fires once.

Not every fader has something to drive in the default program. The default program is SourceSwitch + DropKey + RepeatDelta, so the repeat count, source switch, encoder, mosh, order, tape and preset controls are live immediately. The Stutter, Bloom and Corrupt faders become active when a preset carries those ops; the Pulse faders need the rhythm layer, and the motion pots need motion modulation.

Bindings live in assets/control-map.json; edit the map and relaunch, with no rebuild. The map is validated strictly, so a bad binding is reported rather than silently ignored.

Troubleshooting

Specifications

Credits

MoshBox's behaviour is informed by Interstream v2.0 by Jonas Bers / Signal Culture.