# WC3 Classic D3D9 command stream This directory contains the experimental client-GPU renderer. It is not a production transport yet. The current production H.264 path remains the fallback and the default. ## What the proof established A real Warcraft III 1.28.5 replay was captured through D3D9. The exact browser player renders at a 40 FPS presentation cap. It uses a 4:3 WebGPU target that matches the browser's physical display pixels, applies 1.25x client-side supersampling, and caps the backing target at 2560 pixels wide. The upgraded lab asks Classic to generate its D3D9 UI at an authoritative 1600x1200. It keeps the exact 4:3 aspect ratio. The website can letterbox that target inside a 1600x800 or 1600x900 viewport. It must not stretch the game horizontally. The measured browser render time was approximately 0.9 ms per frame on the test Mac. The accepted lab profile uses `W3_D3D9_HEADLESS_FAST=1`, a 40 FPS game cap, and a 40 FPS command-capture cap. It records the D3D state before bypassing WineD3D state forwarding, draw rasterization, and `Present`. The earlier 1024x768 profile measured approximately 0.22-0.23 vCPU in total. The final native 1600x1200 lab sample used approximately 0.126 vCPU for Warcraft plus the recorder and 0.091 vCPU for the C++ WebRTC and Opus relay. This is about 0.217 vCPU before small Wine helper costs. The Python relay remains only as a reference fallback. Production still needs the planned pack-cache work. An earlier XOR-mask bundle measured about 10.7 KiB per compressed frame. The live lab now sends one reliable fixed geometry base per draw slot. Every disposable frame contains XOR-mask deltas from those fixed reliable bases. A dropped frame therefore cannot poison later frames. Base refresh is demand-driven: the recorder tracks the wire cost of the deltas each slice emits against its stable base and re-sends the exact base on the reliable plane once the accumulated deltas cost more than half of it. A per-slice probe backs the refresh spacing off for motion-dominated geometry that a fresh base cannot make cheaper, and a per-frame byte budget keeps rebases from bursting on the reliable plane. The old fixed 600-frame stagger remains as a backstop. Before this, bases aged with the fixed interval, deltas grew, frames got heavier, the adaptive controller lowered FPS, and lower FPS made the interval even longer in wall time - a compounding loop that pushed a 100 ms WAN seat from 40 FPS into sender backpressure at 15-20 FPS. A final direct-UDP browser sample delivered a 24.6 ms median frame gap and a 50.7 ms p95 gap at the 40 FPS source cap. The wire rate varies with scene complexity and remains above the production gate in busy scenes. The initial resource snapshot also remains too large for production. Production must store immutable texture and map resources in a content-addressed pack. Browsers must fetch and cache that pack before the seat switches from video fallback to command mode. The original proof source is in `poc/`. The 1.16 GB source trace is stored in project object storage. Large trace and bundle files are not committed. ## Production shape The design uses four independent paths: 1. The D3D9 proxy records resources, state changes, draws, and `Present`. 2. A local relay reads the recorder without blocking Warcraft's render thread. 3. A native C++ relay sends reliable resources and lossy, independently decodable frames through WebRTC DataChannels. 4. The browser renders on WebGPU. XTEST carries input. A separate WebRTC peer carries 48 kHz stereo Opus from the WC3 PulseAudio monitor. The production design keeps the current video track as a fallback. A command-mode client must prove WebGPU support and render a validated frame before the worker stops live video capture. Safe mid-session restoration of WineD3D state is not implemented yet, so command mode cannot replace the production video path until that fallback test passes. The experimental live client now includes: - bounded envelope parsing, DEFLATE decoding, and two-channel reassembly; - a reliable-resource sequence dependency on every disposable frame; - XOR and compact XOR-mask content-blob reconstruction; - bounded D3D9 resource and fixed-function state reconstruction; - a WebGPU renderer with one batched uniform upload per frame and dynamic uniform offsets; - GPU BC1/BC2/BC3 texture upload when the browser supports it, with a checked CPU decoder fallback; - a `commandReady` edge only after one complete frame renders; - exact 40 FPS presentation with no speculative geometry interpolation. The modular live renderer reproduced the proof scene at 1600x1200 with about 460 draws and a measured 2-4 ms state-build plus GPU-submit time per frame. This test used saved records through the live renderer interface. The direct browser-to-worker WebRTC canary also passed. Production routing remains unchanged because the wider corpus and fallback gates have not passed. ## Transport rules - Never block the game thread on disk, a socket, compression, or WebRTC. - Keep a bounded single-producer/single-consumer queue in the proxy. - Drop a whole command frame when the queue is full. Do not drop part of one. - Send static resources and texture updates on a reliable ordered channel. - Send one reliable fixed base for each geometry slot. Put independent frame deltas against those fixed bases on the disposable frame channel. - Send frame fragments on an unordered channel with a 250 ms packet lifetime. A frame spans several SCTP messages. Expiry prevents stale fragments from occupying the send window after a burst, while limited recovery still helps paths near the 100 ms product target. Keep frames unordered to avoid cross-frame head-of-line blocking. - Keep the disposable frame send window near the measured bandwidth-delay product. The current limits are 256 KiB to 1 MiB. A large generic SCTP buffer converts congestion into visible latency even when frames expire. - Prefix each frame with the last reliable resource sequence it depends on. Drop the frame if that sequence is not processed within 100 ms. - Keep each application fragment at or below 60 KiB. This stays below the GStreamer SCTP message cap after the W3CS envelope. - Start every frame with a complete small D3D state snapshot. - Tag each resource reference with its generation. - Skip a frame with a missing resource generation. Request that resource over the reliable control channel. - Bound every count and allocation before parsing untrusted network data. See `WIRE-PROTOCOL.md` for the envelope and safety limits. ## Release gates Command mode stays behind `W3_D3D9_COMMAND_STREAM=0` until all gates pass: - exact side-by-side images pass on at least 20 maps and all supported Classic patch families; - clicks, selection, camera movement, replay speed, pause, and seek still map to the authoritative server camera; - median wire rate is below 5 Mbit/s and p95 is below 8 Mbit/s at 40 FPS in the validation corpus; - capture plus relay CPU remains below 0.25 vCPU after startup on the serving instance family; - common assets use a content-addressed browser cache and do not send a 40-plus-MiB snapshot for every new viewer; - loss, reconnect, resource repair, WebGPU device loss, and fallback tests pass; - a new checkpoint profile records the proxy and protocol identity in its manifest. Older checkpoints fail closed for command mode. ## Local tests Run: ```sh python3 -m unittest \ experiments/d3d9-command-stream/test_protocol.py \ experiments/d3d9-command-stream/test_relay.py npm test --prefix experiments/d3d9-command-stream/web ``` ## Disposable end-to-end lab The lab launcher deploys the bridge scripts to an existing disposable Classic worker, starts its replay supervisor, opens the local page server, and keeps an SSH tunnel alive. The worker must already contain the Classic runtime, recorder DLL, and selected replay. ```sh export W3CS_LAB_HOST=ubuntu@203.0.113.10 export W3CS_LAB_KEY=/absolute/path/to/lab.pem export W3CS_LAB_REPLAY=/home/ubuntu/w3cs-lab/test.w3g # Optional; these are the native-HD defaults. export W3CS_LAB_SOURCE_WIDTH=1600 export W3CS_LAB_SOURCE_HEIGHT=1200 experiments/d3d9-command-stream/poc/run-e2e-lab.sh ``` Each browser page load creates a random session ID. The relay stops the old game, clears the capture, and launches one new authoritative game. A reload therefore receives a complete resource bootstrap and does not leak another Warcraft process. Page teardown closes both WebRTC peers, the signaling socket, the WebGPU device, and the input state. If a browser session disappears, the native relay releases the game. The live page extracts the WC3 cursor atlas from the command stream and draws the race-specific hand locally at the browser's display rate. Normalized hover, click, minimap drag, wheel, and camera-pan events go to the authoritative WC3 window. The server cursor rests at the position the last input chose, exactly like the native game; the browser suppresses WC3's drawn cursor by decoded-pixel checksum, not by one texture id. The game renders the pointer as the final draw of each frame out of a 256x128 cursor atlas (hand frames, attack crosshair, the cyan select reticle, no-entry, scroll arrows), and one session can hold the same atlas content under several texture ids. Every texture whose decoded level 0 matches a known atlas checksum is suppressed (web/warcraft-cursor.js POINTER_OVERLAY tables); suppressing only the first matched id left the second copy drawing a resting cyan reticle over hovered units. The pointer is never parked outside the window: windowed WC3 clamps an outside pointer to its nearest window edge for edge-scroll, which pinned the camera into a map corner after every click. Each sparse input has an identifier and travels over both the low-overhead control DataChannel and the signaling WebSocket. The relay applies the first copy and discards the duplicate. This keeps the UDP path fast and preserves an independent fallback. The WebGPU canvas uses a 1.25x supersampled physical-display target up to 2560x1920. This prevents a second browser enlargement from softening the image. True fullscreen contains the game in a centered 4:3 surface and adds black bars on wider displays. UI glyphs use linear magnification to reduce stair-stepping and avoid video-codec artifacts. The lab's 1600x1200 source also lets Classic generate sharper screen-space text than the former 1024x768 surface. Glyphs remain Classic bitmap assets rather than replacement fonts. The native relay applies backpressure only to its recorder-file reader when the reliable queue is full. It does not block Warcraft's recorder thread and does not discard resource or dynamic-texture records. Disposable frame traffic keeps a latest-frame queue. GStreamer is not in the Direct3D command codec or WebGPU renderer. The C++ relay owns framing, checksums, Zstandard coding with frame dictionaries, queue policy, latest-frame selection, and XTEST input. GStreamer supplies its mature ICE, DTLS, SCTP DataChannel, RTP, and Opus plumbing. This keeps the hot command path small while avoiding a custom WebRTC implementation. ## Fidelity enforcement The recorder now fails closed instead of diverging silently. Every D3D9 device path the stream cannot represent is hooked: UP draws, patches, state blocks, `MultiplyTransform`, `Reset`, clip planes, shaders and declarations, render-target changes, `StretchRect`, `ColorFill`, cube and volume textures, gamma ramps, instancing, out-of-range stages and lights, unsupported texture formats on a sampled stage, multi-stream draws, and clear shapes the browser cannot reproduce. The first use of each path sends one reliable `ERROR` message and the browser drops to the video fallback with the method name. The browser renderer counts its own approximations per draw (for example blend operations, stencil, stage-1 combiners, fog modes, or border filtering) and reports them in the stats overlay as `fidelityViolations`. The lab page accepts three parity switches: - `?parity=1` renders at exactly the authoritative source resolution with D3D9 pixel-center placement and captured sampler filtering, then scales the finished image to the canvas; - `?strict=1` turns renderer fidelity violations into hard errors; - `?exactOrder=1` applies reliable resources in exact frame-dependency order, so a texture update cannot overtake the frame that referenced the older generation. `W3_D3D9_EXACT_GEOMETRY=1` on the worker disables the tolerance-bounded float16 vertex deltas, so every transmitted geometry byte is exact. The only remaining lossy path is then linear UI magnification, which parity mode also disables. A dropped recovery can no longer freeze a session: the recorder regenerates a failed keyframe on the next captured frame, and the relay re-requests a stalled recovery through the game-control nonce every five seconds instead of ignoring repair requests while one is outstanding. A resource record that fails to apply (for example a texture blob pruned before its reference arrived) requests a recovery with reason `resource-apply` instead of killing the transport — the next recovery re-delivers complete resource state. Only non-state errors (reassembly, decompress, cache I/O) still fail closed. When the transport does fail, the viewer reports the root cause to the relay over the still-open control channel (`native viewer stream error: ...` in the relay log), shows it in the status line, and reloads itself after 3 seconds (at most 3 restarts per 5 minutes). A viewer is stateless and the pack-cache makes a rejoin cheap, so a fatal stream error is a short blip instead of a permanent freeze. ## Interactive seat behavior The lab page captures the mouse with the Pointer Lock API on the first click, like the native game: the locally drawn WC3 hand is clamped to the game surface, so edge panning engages reliably even on fast flicks. Esc releases the lock and the next click recaptures it; `?lock=0` falls back to the hover model. The hand renders at its native pixel size like an OS cursor (`?cursorScale=` to adjust). Auto Camera now defaults OFF on an interactive seat because it yanks the view to the action after every manual click or pan, and its constantly moving camera also inflates every frame's geometry delta; `?autoCam=1` restores passive follow-the-action viewing. Every manual click, wheel, or pan disables it. ## Resolution policy One session has exactly one authoritative game resolution. The page requests it in the `session` message (`?res=800x600|1024x768|1280x960|1600x1200`, default 1600x1200), the relay validates it and treats it as part of the warm engine identity, and the launch script stamps it into the game registry immediately before every launch. The page also cross-checks the real game window size from input acknowledgements against the command stream's viewport and reports any mismatch. There is no dynamic resolution switching: a change restarts the replay, so it is a manual choice. Slow networks are handled by the adaptive frame-rate controller, not by resolution — the dominant wire cost is world geometry, which is resolution-independent. With the tunnel and remote bridge running, validate two independent session bootstraps and one reload replacement: ```sh node experiments/d3d9-command-stream/poc/e2e-session-canary.mjs ``` The August 18 native-HD canary passed a clean 1600x1200 session over direct UDP WebRTC. A 240-frame steady window measured a 24.6 ms median gap, a 50.7 ms p95 gap, and a 73.6 ms maximum gap. The browser received a live 48 kHz Opus track at approximately 96 kbit/s with zero packet loss in the final sample. Exact 1600x1200 click mapping, camera-pan key press and release, and centered 4:3 fullscreen also passed. This is still a disposable lab. It is not production routing or an AMI/checkpoint promotion. The latency-boundary canary used the 800x600 no-raster profile over a measured 105-107 ms RTT. It reached and held 40 FPS. During more than two minutes at the cap, the disposable-frame buffer stayed at zero in each relay sample and did not enter backpressure. A click acknowledged in 106 ms. Audio continued on the Opus peer, and fullscreen stayed centered at a measured 1.33333 aspect ratio. The product policy targets 100 ms and admits a stable median through 110 ms to avoid flapping. It rejects consistently more-distant workers before a replay seat is claimed.