Audio validation

This page records how each audio/UI change is validated: what the automated regression tests cover, and which listening, device and visual checks still have to be done by hand in a running game. Automated coverage proves signal behaviour and contracts; it does not replace real devices, real rooms or real networks.

Headphone spatial playbackCopy Anchor Link

HRIR convolutionCopy Anchor Link

Playback now convolves each positioned mono stream with the horizontal MIT KEMAR diffuse-field HRIRs (Bill Gardner and Keith Martin, MIT Media Laboratory, 1994).

• Adjacent 5-degree measurements interpolate, and filter/gain changes ramp sample by sample across each 20 ms frame.
• Each voice retains its own FIR history across packets and clears it on stream reset, decoding failure or silence after draining the tail.
• The output device buffer request is reduced from 60 to 40 ms; the device may choose a different actual size.
• The echo reference still uses the final stereo mix and actual queued device samples.

BinauralSpatializerTest checks Minecraft direction signs, distinct front/rear responses, interaural delay and head shadow, mirrored ears, a rapid 180-degree turn against a continuous crossfade reference, cross-frame tails, reset and unpositioned mono.

Manual listening remains required:

1. Have a stationary player speak in front, behind and to each side.
2. Turn slowly, rapidly and across the yaw wrap boundary.
3. Check continuous speech for clicks/dropouts.
4. Test multiple simultaneous speakers and mute/deafen.
5. Use stereo headphones with other surround processing disabled for the baseline.
6. Also check the shorter output buffer on the actual device.

These automated tests prove signal behavior, not subjective 3D quality.

Echo cancellation recoveryCopy Anchor Link

Echo cancellation now rejects a processed frame when its residual energy exceeds the original microphone energy, preserves the original samples and clears the learned weights. This prevents a stale acoustic model from injecting inverted speaker audio into a silent microphone and lets the filter relearn without an off/on toggle.

A deterministic regression failed before the fix on a trained filter with continuing playback and a silent mic; it now passes, including recovery to a changed acoustic response and the existing startup alignment tests. This reproduces a possible mechanism, not the affected player's device conditions.

Device validation:

• Start with cancellation already enabled, have another player speak, then change speaker/headphone routing and verify recovery without toggling.
• Also check simultaneous local/remote speech: the energy guard may temporarily pass through real echo while the model relearns.

Nameplate hookCopy Anchor Link

Nameplate rendering hooks the concrete PlayerEntityRenderer.renderLabelIfPresent before its matrix pop: Minecraft 1.21.11 does not call the base renderer's label method for players. This retains scoreboard-line translation and the special extra-ears vertical offset.

PlayerNameplateHookTest checks the real Minecraft bytecode's injection site and the configured mixin; draw-helper tests alone did not catch the previous missing hook. This is automated contract proof, not an in-game visual check.

Voice activation pre-rollCopy Anchor Link

• Automatic voice activation retains the last three unsent processed frames (60 ms) and sends them in order when speech opens, with a single stream-start marker.
• Quiet-ending hangover increases slightly from 240 to 300 ms; detection thresholds and receiver buffering are unchanged.
• Pre-roll is cleared when muted/deafened, outside automatic activation, and for mic tests; PTT release remains immediate.

Tests cover bounded owned samples, FIFO/exact-once drain, clearing, 300 ms cutoff and immediate mute/PTT stop. Actual onset/ending listening comparison remains a manual check; this targets VAD clipping, not every possible codec/network gap.

Automatic reconnection and fresh tokensCopy Anchor Link

Token and retry behaviourCopy Anchor Link

The client reuses the existing socketsvc:handshake_request plugin channel. After a transport/authentication failure it requests a new one-use token before opening another WebSocket; it never caches/reuses the failed token.

Established connections do not need periodic token refresh or disruptive reconnects.

When retries run and stopCopy Anchor Link

• Automatic requests run on the client game tick only when the current Minecraft connection advertises the refresh channel. Servers without the plugin settle to offline silently.
• Missing token responses retry; leaving/changing game servers clears the schedule, and delayed payloads from the previous network handler are ignored.
• Handshakes received while a socket is connecting/connected are ignored instead of replacing a healthy attempt.
• Explicit replacement, quit, rate/policy and unsupported-message rejections (4000/4003/4008/1008/1003) stop automatic retries; /ssvc reconnect can explicitly start again.
• Transient errors and 4001 token rejection retry automatically.
• Existing WebSocket heartbeat detection determines when an otherwise silent broken transport is reported; retry delay starts then.

State and notificationsCopy Anchor Link

Mute/deafen and first-use privacy settings survive recovery. Repeated failures notify chat only once until stable recovery, with an automatic-retry hint in settings.

Regression tests cover backoff/cap/stable reset, no-plugin behavior, lost responses, fresh request dispatch, throttling, session changes, terminal close policy, retained mute/deafen and notification suppression. Actual network drop/recovery testing in a running Minecraft client remains pending.

Build and automated proofCopy Anchor Link

Install JDK 21 and run (the committed daemon criteria and Java toolchains select 21 even when the launcher uses JDK 17):

.\gradlew.bat :client:build :server:build

The client's check includes audioRegressionTest, a standalone test harness without a running game or audio device. It covers PTT/mute priority, voice hangover, initial/recovery buffering, bounded backlog, gain limiting, echo reference expiration, native Opus roundtrip, native RNNoise noise reduction, automatic delay confidence, correlated echo reduction, independent local speech preservation and roster parsing.

JUnit regression tests additionally cover malformed settings recovery/backup, atomic replacement, invalid enum/volume normalization, invalid endpoints, deafen/mute invariants, zero-length microphone reads, mixed-validity rosters, same-world relay, departed senders, token revocation/expiry, inbound frame limits, rolling rate limits and slow receivers. These tests do not start Minecraft or a live Paper server.

InstallationCopy Anchor Link

Client languagesCopy Anchor Link

The client follows Minecraft's language setting, with built-in English (en_us) and Traditional Chinese (zh_tw). Other locales use Minecraft's English fallback. Settings, tooltips, narration labels, keybinding names, client command replies and connection/processing messages use language keys. Player/device names stay unchanged; server administration messages and diagnostic logs remain English.

Catalogs live in client/common/src/main/resources/assets/socketsvc/lang/. To add a language, copy en_us.json to the Minecraft locale filename, translate values and preserve keys and %s parameters. No custom language selector or configuration migration is needed. Check the three tabs at small GUI scales in both languages in-game; automated tests verify key parity, references, fallback and formatted messages, not visual layout.

Installation notesCopy Anchor Link

• Client and server artifacts: see the artifact table in Installation (per Minecraft version and loader; the 0.1.0 paths client/build/libs/ and server/build/libs/ are historical).
• Update both ends to enable the connected-player list. Older servers still carry voice audio but cannot provide that list.
• RNNoise native resources are bundled. Cloth Config is no longer required.
• Existing saved settings are preserved. If echo cancellation was previously saved as off, enable it in Sound settings; new configurations default to automatic echo alignment.
• First-run setup connects immediately but does not transmit microphone audio until Finish. /ssvc reconnect requests a fresh token without leaving Minecraft (requires an updated server; token requests have a five-second cooldown).
• For cloudflared on the same machine, set bind-address: "127.0.0.1". The default remains 0.0.0.0 for direct network access. Changing the bind address or port requires a restart; a listener failure logs a severe error and disables the plugin.
• socketsvc-server-0.1.0-plain.jar is only the unshaded build artifact; do not install it.

In-game acceptance still requiredCopy Anchor Link

The settings and first-use screen now share a centered, solid-color dialog capped at 360 x 286 GUI units. Three top tabs replace the sidebar:

Smaller viewports scroll the contents without moving the tabs or footer. Check all three tabs at 320 x 240, 480 x 270 and 960 x 540 GUI sizes; long device lists, keyboard navigation and roster updates during slider dragging must remain usable. The General tab deliberately omits gain/delay/threshold diagnostics. This redesign has build/regression coverage but has not been visually verified in a running game.

Scope and limitsCopy Anchor Link

Developer overlayCopy Anchor Link

Developer mode is opt-in under Advanced settings and persists in the client config. The top-right overlay refreshes twice per second and shows:

• local TX-queued/RX packet rates and cumulative counters
• local TX drops
• playback queue depth/overflow
• decoder failures
• dry-buffer transitions
• microphone level
• time since the last audio packet
• negotiated v1/v2, sequence gaps, successful PLC frames and stale/duplicate frames

Counters reset on new connection attempts and remain available after failure. A successful TX count means accepted into the WebSocket send queue, not acknowledged delivery. Dry-buffer transitions include ordinary speech pauses; RX silence can also mean nobody nearby is talking. Sequence gaps are playout/application gaps, not true TCP network loss; end-to-end latency and remote server drop telemetry remain unavailable in the HUD. No tokens, server URLs or player identities are displayed.

Playback bufferingCopy Anchor Link

Playback adapts timed prebuffering from 60 to 400 ms, with a hard 50-frame (1 second) backlog cap per speaker to accommodate catch-up bursts during recovery. A burst's packet count no longer bypasses the timed wait. Arrival gaps raise the target rapidly; a two-second hold and gradual decay avoid forgetting the stall as its backlog arrives.

The target decides where playback starts after a stall. A separate catch-up decides how deep the queue is then allowed to stay, because depth never falls on its own: the mixer pulls one frame per 20 ms and the speaker sends one per 20 ms, so whatever a burst or a stall left queued used to be carried for the rest of the session. In simulation, a link that stalled once and then ran perfectly for half a minute sat 220 ms deep while the target read a healthy 60 ms.

Playback now hands frames back once the queue is more than two frames past the target, at one frame per 240 ms (about an 8% speed-up, the most that passes unnoticed in speech), or one per 80 ms while more than eight frames behind. A skipped frame takes its sequence with it, or the concealment path reads the skip as loss and synthesizes a replacement, undoing the trim. A queue that is not behind is never trimmed, so a healthy link is untouched and stable links still return to low-delay starts. The overlay counts these separately from overflow drops, as latency trims.

The target itself also comes back down sooner: a gap re-arms the hold only when it genuinely exceeds the current target - previously any gap the buffer was already sized for re-armed it, so the estimate could only rise - and it then decays one frame per 250 ms after a one-second hold rather than one per second after two.

Tests cover stable 60 ms starts without frame loss, 480/120 ms burst recovery without subsequent starvation in the simulation, the previous 180/60 ms burst scenario, a 200 ms stall's standing delay being given back over ten seconds of clean unbroken speech without any frame reading as lost, and a link with no backlog never being trimmed at all. Arbitrarily long TCP stalls cannot be hidden with finite buffering; real cross-region testing remains required.

Instrument captureCopy Anchor Link

The same standing-delay problem existed at the instrument's capture end, independently of the network. The instrument stream holds the pipeline's 20 ms timeline with a timer, and a line-in is opened with an eight-frame (160 ms) hardware buffer; reading one frame per tick keeps a backlog exactly as deep as it got, so a single moment of thread starvation was worth up to 160 ms of added delay for as long as the input stayed open.

PcmSource now reports whole frames already waiting, the mixer reports the furthest behind any one source is, and the stream reads straight on instead of parking while that is non-zero; the device's own blocking read takes the pacing back the moment nothing is waiting. A generated source (a synth, the drum machine, a loop) always reports zero and stays paced entirely by the timer. The microphone path never had this problem: its capture loop has no timer of its own, so it already drained at whatever rate the device could supply.

Backpressure queuesCopy Anchor Link

Both client and server now retain audio during temporary WebSocket backpressure in an application-owned queue.

• Oldest packets are discarded on overflow/expiration, with local TX drops in the client overlay and aggregate relay drops in /ssvc-server status (including closed/revoked queues).
• The dedicated daemon retries even after capture stops; disconnect/shutdown clears pending data.
• Only send when the WebSocket queue has drained, so it does not become an unbounded second backlog. Bytes already handed to TCP/OS buffers cannot be retracted or age-limited by this queue.

Tests cover ordered recovery, input ownership, per-speaker/aggregate caps, expiration and revocation. The negotiated v2 format adds speech stream IDs and sequences with a v1 fallback; see Voice protocol for mixed-version rollout and rollback. Confirmed gaps of 1–3 frames use native Opus PLC (at most 60 ms). Empty queues do not synthesize speech; longer holes reset the decoder and resume real audio.

Player nameplate iconsCopy Anchor Link

Player voice icons render independently four label pixels after the original name's right edge (nameWidth / 2 + 4). The original name text, centering and background width are untouched, so toggling an icon cannot shift the nametag. Icon text commands explicitly use zero background/outline and no shadow, with normal depth testing, the vanilla label transform and per-render-state storage. Existing speaking/disconnected/speaker-off textures are reused via a private bitmap font.

• Priority is disconnected, deafened, speaking, then hidden; unknown roster or local disconnection does not falsely mark everyone offline.
• Speaking reflects locally played audio.
• Deafened status is synchronized through the separately negotiated listener-state extension; update both ends for this status.
• Advanced → Interface → Player nameplate voice icons still controls all icons independently.

Tests verify unchanged name text, fixed x offset across icon states, zero background, matrix restoration, hidden labels, capability fallback, toggle coalescing, server relay suppression/resumption, status cleanup and rate limits. In-game visual verification remains pending: check vanilla name hiding/team rules, sneaking, scoreboard/team prefixes, F1, resource packs, and multiple GUI scales. No TAB/chat names are modified.

Server routing: fade and spectatorsCopy Anchor Link

Server routing now supports fade-start-distance (blocks, from zero up to but excluding hearing-range) and spectator-interaction (default false).

• New configs fade from zero; existing configs without an explicit fade start keep the old 70% start, ignoring bundled defaults.
• Invalid fade values use that legacy start; invalid hearing ranges disable relay.
• /ssvc-server reload atomically applies these settings, and status reports effective values.
• With spectator interaction off, spectators may hear active players and talk to other spectators, but active players cannot hear spectators. World and distance checks still apply.
• Game mode is sampled each server tick; already queued audio is not retroactively revoked.

HUDCopy Anchor Link

The HUD uses this priority: disconnected, incoming voice disabled, microphone muted, speaking; connected unmuted idle is hidden. It remains unframed and respects F1, the HUD setting and open screens. Toggle artwork is selected by actual image contents: the historical mic.png is a speaker, while speak.png is a microphone. Verify these icons and spectator mode transitions in-game.

Server protections and remaining gapsCopy Anchor Link

• The server rejects inbound audio larger than 1275 bytes.
• Its token bucket permits 60 packets/s sustained with up to 60 packets of burst credit, accommodating delayed normal 50 fps arrivals while bounding floods.
• Only the latest unsent roster is retained for retry.

These protections do not eliminate operating-system/TCP buffering or retransmission stalls. The adaptive echo filter is not WebRTC AEC3; room reverberation, device clock drift, nonlinear speakers and double-talk require real acoustic validation. Groups, whispering, recordings and other parity gaps are tracked in the roadmap.

Review follow-up acceptanceCopy Anchor Link

All fourteen reported items have corresponding code/configuration changes. Still verify in-game:

• Spend over 30 seconds in first-run setup.
• Expire a token and use /ssvc reconnect.
• Unplug the microphone.
• Disconnect while transmitting.
• Drag a player volume slider while another player joins/leaves.
• Exercise a Floodgate-prefixed player name.
• Also test an occupied bind port on a real Paper server.

Automated tests cover the underlying logic, not these complete platform interactions.

ReferencesCopy Anchor Link

• Simple Voice Chat microphone processing
• Simple Voice Chat audio channel
• RNNoise4J API and platforms

Only the processing approach and published RNNoise library are reused; the Simple Voice Chat networking architecture has not been imported.