Soundboard design

The soundboard lets a player trigger a local sound clip that's transmitted to nearby players as its own audio channel, independent of their microphone: muting the mic does not silence it, but a listener's own deafen state still blocks it, same as it blocks mic audio. It is gated server-side by a permission node (socketsvc.soundboard) so it can be blocked per-player/per-group regardless of what the client does.

This page covers the client side and the design history. For players, see the Soundboard guide.

Client architecture複製段落連結

Send複製段落連結

A triggered clip is decoded to 48kHz mono 16-bit PCM via javax.sound.sampled.AudioSystem:

• WAV decodes with zero extra dependencies (the JDK's built-in reader, same approach the server's PcmSource.wav() already uses).
• MP3, OGG and FLAC decode through the same AudioSystem call once com.googlecode.soundlibs:mp3spi:1.9.5.4, :vorbisspi:1.0.3.3 and org.jflac:jflac-codec:1.5.2 are on the classpath, since all three register themselves as standard javax.sound.sampled SPI providers — one decode path for all four formats, not four separate decoders.

PCM is fed through the existing OpusCodec.Encoder (same encoder the mic path already uses) in 960-sample/20ms frames, sent via a new paced sender tagged with C2S kind 1. Each concurrent session owns its stream ID and sequence counter, so interleaved clips cannot reset one another.

It never touches AudioCapture's mute gate at all — mute is enforced purely by AudioCapture choosing not to call the mic send path, so a soundboard trigger calling its own independent send path bypasses it with no special-case code anywhere.

Receive複製段落連結

VoiceWebSocketClient/AudioPlayback recognize S2C kind 2 and mix it through a new, independent per-listener control — SocketSvcConfig.soundboardPlayerVolumes — instead of the existing playerVolumes used for kind 0, while still passing through the listener's own single local deafened check (already gates the whole incoming pipeline, both kinds).

Library複製段落連結

SoundboardManager scans FabricLoader.getConfigDir()/socketsvc/soundboard/ recursively; each subfolder is a category. Favorites, local/player volumes, loop flags, custom start points, and keybind-slot assignments are persisted by relative path.

UI複製段落連結

SoundboardScreen is built on the existing ModernVoiceScreen framework, same as every other SocketSVC dialog:

• Navigation: breadcrumb + list navigation per folder level (a .. row to go up, folder rows to descend). No tree widget exists anywhere in this codebase, so this mirrors IconInboxScreen's flat-list-per-level style rather than inventing one.
• Sound rows: each row shows name, favorite, loop and keybind-slot controls, an inline local/player volume slider, and click-to-play.
• Search scans all subfolders.
• Now-playing transport: a persistent transport provides pause/resume, stop, seek, five-second skips, looping, and start-point capture.
• Screen-level controls: synchronized volumes, sender-side master volume, and opt-in concurrent playback.
• Open folder: a button uses Util.getPlatform().openPath(...) — the same API IconInboxScreen already uses for exactly this, deliberately not java.awt.Desktop (documented AWT/LWJGL windowing conflict on macOS elsewhere in this codebase).
• Per-listener volume: the per-listener soundboard volume/mute control is a row directly under each player's existing mic-volume row in VoiceSettingsScreen's Players tab (reusing ModernSlider.playerHead(...)), not a separate tab.

Keybind複製段落連結

U (confirmed unbound by any existing default) opens/closes SoundboardScreen, registered and consumed exactly like the existing V settings key.

Quick-pick keybind複製段落連結

A second, default-unbound keybinding (key.socketsvc.soundboard_quickpick) opens SoundboardQuickPickScreen on press — a small overlay strip above the hotbar listing favorited clips (SoundboardManager.listFavorites()).

• Scrolling moves the highlight.
• Releasing the physical key (polled every tick via SocketSvcClient.isSoundboardQuickPickKeyDown(), the same "isDown() works regardless of open screen" reliance already used for push-to-talk) plays whichever clip is highlighted and closes the overlay.
• Escape or a left click cancel/confirm without needing a KeyMapping "hold" primitive beyond polling isDown().

yt-dlp download from URL複製段落連結

• yt-dlp binary: YtDlpNative caches yt-dlp itself on demand through ManagedTool (SHA-256 verification of the download before it's used, executable bit set on Unix — see Phase 2 §3). Which version is decided by the update service's /v1/tools/yt-dlp, not pinned in code; see Current code below.
• ffmpeg: yt-dlp's own -x/--audio-format needs ffmpeg for the actual audio extraction and does not bundle it, so FfmpegNative does the same on-demand download+verify through ManagedTool for a static ffmpeg build from yt-dlp/FFmpeg-Builds (Windows/Linux only — that upstream project publishes no macOS builds). A system ffmpeg already on PATH is always preferred and used as-is on any platform, including macOS via e.g. brew install ffmpeg. Without this, every "Download from URL" attempt on a machine without a pre-installed ffmpeg would fail outright with an opaque yt-dlp error.
• Fetch: YtDlpFetcher runs it as a subprocess to pull just the audio out of a player-pasted URL, built as an explicit ProcessBuilder argument list (never a shell) with the URL as a single, final, literal argument. YtDlpFetcherTest verifies the exact argument shape, including that a deliberately shell-hostile URL never produces shell syntax.
• Consent: YtDlpDownloadConfirmScreen gates the first use behind explicit consent (SocketSvcConfig.ignoreYtDlpWarning for "don't ask again", same shape as ignoreOtherVoiceChatModWarning), spelling out both trust-boundary points from Phase 2 §5.
• Screen: SoundboardScreen hosts the URL field + Download button (replaced by a status line + Cancel while a fetch is running); a successful download lands in whichever folder is currently open and the listing refreshes automatically.

Deferred items複製段落連結

Nothing — every item originally listed as deferred (yt-dlp auto-download, FLAC, the hold-and-scroll quick-pick keybind) is now implemented. The yt-dlp piece's original design rationale is in Phase 2 below.

Phase 2: yt-dlp download複製段落連結

Status: implemented. This started as a forward-looking plan for the piece deferred out of Phase 1 (see Deferred items); it's now built (YtDlpNative, YtDlpDownloader — since folded into ManagedTool/ToolDownloader — YtDlpFetcher, YtDlpDownloadConfirmScreen, the URL bar in SoundboardScreen) — see Client architecture for the as-built summary. The sections below are kept as-is (design rationale, not a changelog); where implementation diverged from the original plan in a small way, it's noted inline.

1. Why this is its own phase複製段落連結

Every other piece of this mod that fetches something on demand (DeepFilterDownloader/DeepFilterNative) downloads a fixed, versioned, first-party asset (ONNX Runtime from Microsoft's own GitHub releases; a named DeepFilterNet model) to a known URL baked into the mod. yt-dlp is different in two ways that change the trust/consent story:

• It downloads and executes a separate third-party binary (GPL-3.0 licensed, not ours) as a subprocess, not a passive data file.
• That binary then fetches arbitrary content from a URL the player supplies, from whatever site they paste — not a single fixed, audited upstream.

Both of those are meaningfully bigger trust boundaries than anything this mod does today, so this needs its own explicit, worse-case-labeled consent screen (see §5) rather than reusing DeepFilterDownloadConfirmScreen's wording as-is, even though the mechanics below intentionally mirror that existing download machinery closely.

2. Scope複製段落連結

In scope:

• Download the yt-dlp binary itself, on demand, per platform (mirrors DeepFilterNative's on-demand-download shape).
• A "Download from URL" entry point in SoundboardScreen (paste a URL, hit download) that runs yt-dlp as a subprocess to fetch just the audio into the soundboard folder in a supported format.
• Progress, cancel, and error reporting for that subprocess.
• A one-time (dismissible-forever, like OtherVoiceChatModWarning's "don't warn me again") consent screen before the first download, spelling out both trust-boundary points from §1.

Out of scope (unchanged from the Phase 1 deferred list, still deferred beyond this phase too):

• The hold-and-scroll wheel quick-play keybind.
• FLAC playback support (yt-dlp can output FLAC; whether it's offered as a format choice here depends on that separately-tracked decision, not on this plan — see §7).
• Any kind of built-in search/browse UI for content — this is strictly "paste a URL you already have," the same scope as running yt-dlp yourself from a terminal, never a discovery feature.

3. Downloading the yt-dlp binary複製段落連結

Mirrors DeepFilterNative/DeepFilterDownloader's shape closely: a YtDlpNative-equivalent status/cache class, a downloader that uses java.net.http.HttpClient (already the only HTTP mechanism used anywhere in this codebase — no new dependency), a Status enum (NOT_DOWNLOADED/DOWNLOADING/READY/FAILED; no UNSUPPORTED_PLATFORM needed since yt-dlp publishes binaries for every platform this mod supports), and a .complete marker file gating "ready," same as DeepFilterNative.cacheDir().

Differences from the DeepFilterNet case:

• One binary, not an archive to extract. yt-dlp's GitHub releases publish a single executable per platform (yt-dlp for Linux/mac, yt-dlp.exe for Windows, plus separate yt-dlp_macos/yt-dlp_linux_aarch64 variants) — download straight to cacheDir().resolve(binaryName()), no zip/tar extraction step like the ONNX Runtime archives need.
• Must be marked executable on Unix. DeepFilterDownloader never needs this (it downloads a shared library loaded via System.load, not run as a process) — after download, file.toFile().setExecutable(true) on mac/Linux (a no-op, harmless, on Windows).
• Pin an exact version, same principle as ONNXRUNTIME_VERSION. Track a specific yt-dlp release tag, not "latest," so behavior doesn't shift under players without a mod update; bump it deliberately in a future commit the same way ONNXRUNTIME_VERSION/MODEL_BRANCH would be bumped.
• Verify the download. yt-dlp publishes SHA256SUMS per release; check the downloaded binary's hash against that before marking .complete (DeepFilterDownloader doesn't currently do this for the ONNX assets — worth doing here regardless, since this asset is an executable, not a passive library).

Cache location: FabricLoader.getConfigDir()/socketsvc/ytdlp/<platform>/, same parent (socketsvc/) and per-platform-subdirectory convention as socketsvc/deepfilter/<platform>/.

4. Downloading a clip複製段落連結

New class, e.g. YtDlpFetcher, invoked from a background thread (never the render thread, same rule as everywhere else audio/file work happens in this codebase):

5. Consent screen複製段落連結

New YtDlpDownloadConfirmScreen (structurally identical to DeepFilterDownloadConfirmScreen — wrapped body text via font.split, Cancel/Download buttons at the bottom of the panel), but its own distinct copy covering both trust-boundary points from §1 explicitly:

• Downloads and runs a third-party program (yt-dlp, GPL-3.0) to fetch audio from the URL you provide.
• SocketSVC has no control over, and does not vet, whatever site or content a pasted URL points to — only download from sources you trust, and only content you have the right to use.
• Shown once, with a persistent "don't ask me again" option (SocketSvcConfig.ignoreYtDlpWarning or similar, following ignoreOtherVoiceChatModWarning's existing exact pattern) — but always shown again if that flag is unset, i.e. it gates the nag, never the download itself.

6. SoundboardScreen integration複製段落連結

• A new row/button in SoundboardScreen, visible in every folder view (not just root) so a download always lands in whatever folder is currently open: opens a small text-input prompt for the URL (no existing text-input widget in this codebase's Modern* widget set — needs a new one, or reuse Minecraft's vanilla EditBox inside a ModernVoiceScreen panel the way VoiceSetupScreen/other screens already mix vanilla widgets with the Modern* framework where needed).
• While a download is running, show progress inline (a slim progress bar row, or repurpose ModernMeter — already used for mic level display — for a determinate 0-100% bar) plus a Cancel action.
• Route the "first ever download" check through §5's confirm screen before actually calling into YtDlpFetcher; subsequent downloads in the same session (or ever, if "don't ask again" was set) skip straight to downloading.

7. Format decision複製段落連結

Original heading: "Format decision (needs revisiting before implementation)".

Phase 1 decodes WAV/MP3/OGG (SoundboardAudio, via the JDK's own WAV reader plus the bundled mp3spi/vorbisspi). yt-dlp itself can produce many formats via --audio-format (best, mp3, m4a, opus, vorbis, wav, flac, ...), but this feature must never let a player produce a file SoundboardAudio can't actually play back — so --audio-format here should be fixed to mp3 unless/until FLAC (or another format) is separately implemented client-side per the Phase 1 FLAC deferral.

Extracting straight to opus and skip re-encoding would be the most efficient option quality/size-wise (yt-dlp's source is very often already Opus-encoded) — worth a follow-up look at whether javax.sound.sampled can be made to decode raw Opus-in-ogg via the existing vorbisspi dependency, or whether that needs its own separate decode path; not resolved in this plan and should be settled before writing YtDlpFetcher.

8. Testing plan複製段落連結

• YtDlpFetcher's subprocess argument-list construction can be unit tested directly (assert the exact List<String> built for a given URL/output folder, without actually invoking ProcessBuilder.start()) — the same "test the command shape, not a real subprocess" approach worth using given this repo's tests don't spin up real external processes anywhere today.
• Filename sanitization (yt-dlp's %(title)s → a safe on-disk name) gets its own focused test table (path separators, .., leading dots, control characters, empty string).
• The download-binary verification step (§3) gets a test against a known good/bad SHA256 pair using a small synthetic fixture, same "build a real fixture, verify against it" discipline as SoundboardAudioTest/GifDecoderTest elsewhere.
• Manual/real-network verification (can't run in this environment): an actual URL download end-to-end, cancel mid-download, a deliberately invalid URL, a very long video (progress bar sanity), running on all three OS families for the executable-bit/binary-selection logic.

9. Open questions複製段落連結

Original heading: "Open questions before starting implementation".

• Exact yt-dlp version to pin (§3) — needs a real decision at implementation time the same way FastStats/Velocity/BungeeCord versions were verified against real Maven metadata earlier in this project's history, not guessed.
• Whether to bundle a yt-dlp.d/config.txt-style fixed argument set (rate limiting, no-playlist, format) that a player cannot override, vs. any player-facing "advanced options" surface at all — current lean is no player-facing raw argument passthrough, ever, given §4's command-injection framing; only ever the fixed, hardcoded argument list.
• Whether output should default to the folder currently open in SoundboardScreen (§4.5) or always to soundboard root with the player free to move it after — current lean is "currently open folder," since it matches how the player was already browsing/organizing when they hit download.

Phase 3: feature parity複製段落連結

Status: implemented. This covers the five gaps identified by comparing our soundboard against XCraftTM/OpenSoundboard's README feature list, after the one real bug that comparison surfaced (yt-dlp needing ffmpeg, which we didn't download or check for) was already fixed directly. Each section below is independently implementable; none of them block each other, though §4 and §5 touch some of the same files.

1. Per-sound custom keybinds複製段落連結

Implemented: ten default-unbound slots, persisted clip assignments, row-level slot cycling, and tick polling through the normal Minecraft keybinding system.

Problem: Minecraft's keybinding system (KeyMapping/KeyBindingHelper) requires every bindable key to be registered once, statically, at mod init — the vanilla Controls screen and the game's key-conflict-context system are both built around a fixed, known-at-startup set of KeyMapping objects. There is no supported way to register a new one dynamically the moment a player drops a new sound file into their folder.

Plan: implement (a). New key.socketsvc.soundboard_slot_<n> bindings (pick a count — 10 mirrors a hotbar and is a reasonable start, easy to raise later since it's just more static registrations), SocketSvcConfig.soundboardSlotClips, slot-assignment UI in SoundboardClipRow/SoundboardScreen, polling in SocketSvcClient's tick handler.

2. Local vs. player volume per clip複製段落連結

Original heading: "Independent local vs. player volume per clip".

Implemented: the compact row control switches between L (local) and P (player/network) volume. Playback uses separate PCM copies and limiter state for the two paths.

Current state (at planning time): SoundboardPlayer.runProducer applies exactly one volume (SocketSvcConfig.soundClipVolume) to each PCM frame before both loopback.play(copy) (what the triggering player hears) and encoder.encode(copy) (what gets sent to everyone else) — one shared value drives both.

Plan:

• SocketSvcConfig gains a second per-clip map alongside the existing soundClipVolumes (which becomes, semantically, "player volume" — what others hear): soundClipLocalVolumes ("what I hear"). Same Map<String, Integer>/clamped-0-200 shape, same sanitize() treatment.
• SoundboardPlayer.runProducer needs two independently-scaled copies of each frame per iteration (one for loopback.play, one for encoder.encode), each through its own AudioLimiter instance — AudioLimiter carries gain-smoothing state (gainForPeak's gain field) that must not be shared between two differently-scaled streams, or one channel's limiting will visibly affect the other's loudness.
• UI: SoundboardClipRow currently has one volume track per clip; adding a second value without doubling every row's height needs a compact toggle (e.g. a small "L"/"P" label button before the track) that switches which of the two volumes the single visible slider/track currently shows and edits — keeps the list as dense as it is today instead of doubling its height. A "synced" mode (OpenSoundboard's own optional simplification, one slider drives both values together) is a reasonable secondary addition once the split itself exists, not a precondition for it. Implemented: the screen-level sync toggle makes edits to either channel update both persisted values.
• Optional, lower priority: a config.soundboardMasterVolumePercent global multiplier on top of every per-clip local/player value, mirroring OpenSoundboard's "global volume multiplier for both channels." Partial equivalent already exists listener-side today (playbackVolumePercent already scales received soundboard audio in AudioPlayback.soundboardPlaybackGain) — this would add the sender-side counterpart. Implemented: soundboardMasterVolumePercent scales both sender-side channels.

3. Search/filter box複製段落連結

Implemented: the persistent search field performs case-insensitive name matching over the flat recursive clip list and reports an explicit no-results state.

Plan: the smallest of the five. Add an EditBox to SoundboardScreen (same construction/re-add-each-rebuild pattern the yt-dlp URL field already established), filtering by case-insensitive substring match on clip name. A non-empty query switches the list to a flat, recursive view (reusing the already-implemented SoundboardManager.listAllClipsRecursively(), same as the Favorites view already does) instead of the current single-folder listing, since a search should find a clip regardless of which subfolder it's actually filed under.

4. Playback controls複製段落連結

Original heading: "Playback controls: pause/resume, seek, skip, custom start point, loop".

Implemented: the now-playing transport exposes pause/resume, stop, five-second skips, a seek slider once duration scanning completes, per-clip start-point capture, and per-clip looping. Durations are cached by relative path and file modification time.

The largest and most architecturally invasive of the five — real rework of the streaming decode/send pipeline, not additive UI.

5. Concurrent playback (toggle)複製段落連結

Implemented: concurrency is opt-in and negotiated-soundboard-only. Sessions are keyed by clip, use independent stream/sequence identities, and feed a clocked local PCM mixer. The legacy mic fallback retains single-clip playback because its protocol cannot represent parallel streams.

Decided: add as an opt-in toggle (SocketSvcConfig.soundboardAllowConcurrent, default off — today's stop-previous behavior stays the default). More invasive than "just a toggle" once traced through every layer that currently assumes exactly one active soundboard voice:

• SoundboardPlayer: currently tracks one current Session field. Needs to become a collection (e.g. Map<String relativePath, Session>, keyed by clip so re-triggering the same clip still restarts it rather than stacking duplicates, while different clips can run simultaneously when the toggle is on). With the toggle off, play() clears the whole map before adding the one new entry — same effective behavior as today.
• SoundboardLoopback: today it's a simple pass-through PCM writer to one output line. Multiple concurrent producer threads writing to that same line directly would interleave rather than mix — audibly garbled, not overlapping — the same class of bug the mic+soundboard rate-limit issue taught us to watch for when two independent streams share one resource. Needs to become a real small mixer (sum concurrent PCM streams with limiting/clipping protection) — conceptually close to what AudioPlayback already does for receiving multiple voices, worth checking whether that mixing logic can be adapted rather than reinvented.
• VoiceWebSocketClient.sendSoundboard: currently keyed to one client-wide soundboardStream/soundboardSequence field pair. Two concurrent clips sharing that single stream identity would corrupt the receiving side's per-stream sequencing exactly like two frame sources sharing one queue slot did earlier in this feature's history (the "same route-tracking slot" note in the original soundboard design doc). Needs a fresh, independent stream/sequence pair generated per concurrent playback session instead of one shared field.

Suggested order複製段落連結

Roughly cheapest/most self-contained to most invasive, though none block each other:

1. (3) search
2. (1) slot keybinds
3. (2) local/player volume split
4. pause/resume half of (4)
5. (5) concurrent playback
6. seek/skip/start-point/loop half of (4)