WC3 CLASSIC CLIENT-GPU GRAPHICS COMMAND STREAM Capture, compression, transport, reconstruction, and WebGPU rendering handoff Document date: 2026-08-19 Repository: /Users/admin/Development/war3replays Base Git revision at the time of review: 7ff53e6 Implementation status: experimental lab only; not production routing 0. PURPOSE AND SCOPE ==================== This document explains the current WC3 Classic client-GPU experiment. It is intended for an engineer who wants to reduce server-to-browser bandwidth without changing the rendered result or making loss recovery less reliable. The two main questions are: 1. What graphics information does the browser receive, and how is it reduced and compressed before transmission? 2. How does the browser decode that information, rebuild the D3D9 state, and render the game with WebGPU? This is a description of the current working tree, not only the older protocol notes. Some older comments and documents describe earlier designs. This file calls out important differences. The experiment does not transmit H.264 video for its command mode. It transmits a compact representation of the D3D9 work that Warcraft III would have given to WineD3D. The browser reconstructs that work and rasterizes it on the client's GPU. The current production website still uses the video path. Command mode remains a laboratory path until visual, recovery, bandwidth, corpus, and fallback gates pass. 1. ONE-PAGE ARCHITECTURE ======================== The data path is: Warcraft III 1.28.5 | | D3D9 COM calls made by the game v d3d9_proxy.dll inside Wine | | length-prefixed W3CS records through a local file/FIFO v native C++ relay on the WC3 worker | | semantic batching + Zstandard + fragmentation v WebRTC DataChannels over ICE/DTLS/SCTP/UDP | | reliable resources and recoveries | partially reliable disposable frames v browser transport and record decoder | | reconstructed resources, blobs, and per-draw D3D9 state v WebGPU fixed-function emulator | v browser canvas at the client's physical display resolution Audio is independent: WC3 PulseAudio monitor -> Opus 48 kHz stereo -> WebRTC audio peer -> browser Input is independent and authoritative on the server: browser pointer/key event -> control DataChannel and duplicate signaling-WebSocket copy -> idempotent C++ relay input handler -> XTest event to the WC3 window -> WC3 changes its camera/selection/replay controls -> the resulting D3D9 command frames return to the browser The command stream is one-way for graphics. Browser input is not a D3D return path. 2. WHAT "D3D9 COMMAND STREAM" MEANS =================================== The browser does not receive raw C++ COM calls or pointers. It receives W3CS records that preserve the parts of those calls needed to draw WC3 Classic. Examples: IDirect3DDevice9::SetRenderState(D3DRS_ALPHABLENDENABLE, TRUE) becomes: opcode 16 SET_RENDER_STATE payload: uint32 state=27, uint32 value=1 IDirect3DDevice9::SetTransform(D3DTS_WORLD, matrix) becomes either: opcode 17 SET_TRANSFORM payload: uint32 transformState + 16 float32 matrix values or the smaller safe form: opcode 34 SET_WORLD_TRANSFORM_COMPACT payload: 13 binary16 matrix values + 3 float32 translation values IDirect3DDevice9::DrawIndexedPrimitive(...) becomes: opcode 33 DRAW_INDEXED_PRIMITIVE payload: primitive type, base vertex, vertex bounds, start index, primitive count, content blob id for the vertex slice, content blob id for the index slice Present is not transmitted as a normal record. Present closes the current logical frame. The recorder sends the accumulated record stream as one W3CS FRAME message. 2.1 D3D9 is bidirectional, but command-mode graphics are not ----------------------------------------------------------- The D3D API has return values, object creation, locks, queries, errors, and other bidirectional behavior. The browser is not a remote D3D device and does not answer those calls. The proxy keeps real Wine D3D9 COM objects and calls the original WineD3D implementation when server rasterization is enabled. The game receives normal D3D return values from WineD3D. In the accepted headless-fast laboratory profile, selected state calls, draw calls, Clear, and Present can return D3D_OK without asking WineD3D to rasterize. The proxy itself maintains the state and resource shadow copies needed for the outbound stream. Object creation and resource access still happen on the server so the game can continue normally. The design therefore does not need a network round trip for each D3D call. Trying to make the browser answer synchronous D3D calls would make a 30-100 ms network path unusable. The current proxy covers a proven WC3 Classic D3D9 subset. It is not a general D3D9, D3D11, DXBC, or arbitrary Windows-game remoting implementation. 3. CAPTURE ON THE WC3 PROCESS ============================ Source: experiments/d3d9-command-stream/native/d3d9_proxy.c experiments/d3d9-command-stream/native/w3cs_protocol.h 3.1 Hooks currently installed ----------------------------- The proxy clones selected COM vtables and hooks these important operations: Resource lifecycle and contents: CreateVertexBuffer CreateIndexBuffer CreateTexture buffer Lock/Unlock texture/surface LockRect/UnlockRect UpdateTexture and surface-copy paths used by WC3 resource Release/destruction SetCursorProperties ShowCursor Small fixed-function state: SetRenderState SetTransform SetTexture SetFVF SetStreamSource SetIndices SetMaterial SetLight LightEnable SetTextureStageState SetSamplerState SetViewport SetScissorRect Clear Drawing and frame boundary: DrawPrimitive DrawIndexedPrimitive Present The proxy tracks up to eight texture/stream stages, eight lights, 256 render states, 512 transforms, 33 texture-stage state types, 16 sampler state types, and 16 texture mip levels. 3.2 The game thread does not do socket or compression work ---------------------------------------------------------- The D3D hook serializes small records into a bounded memory queue. A separate writer thread writes those messages to the local capture file or FIFO. Important current bounds: proxy memory queue: 32 MiB one logical frame: 4 MiB proxy blob cache: 64 MiB proxy native fragment payload: 60 KiB maximum A fragmented native message is queued completely or not at all. The recorder does not publish half a texture or half a geometry blob and then claim that it was sent. If the queue cannot accept a complete disposable frame, that frame is dropped. The game thread does not wait for a network socket or compression operation. The file/FIFO format has a four-byte little-endian packet length before each native W3CS envelope. That four-byte length is local transport framing. It is not sent inside a browser DataChannel message. 3.3 Frame sampling and state deduplication ------------------------------------------ The current lab asks WC3 to run and capture at 40 FPS. The recorder can sample Present calls if WC3 produces more frames than the selected capture rate. State setters are deduplicated. For example, if WC3 sets the same texture, render state, transform, sampler, or material twice, the second identical call does not add another frame record. Every captured frame starts with a complete small-state snapshot. The snapshot contains all currently valid render states, transforms, texture-stage states, sampler states, stream bindings, texture bindings, index binding, FVF, material, lights, viewport, and scissor rectangle. This repeated state costs bytes, but it lets the browser decode a later frame after an earlier disposable frame is lost. A bandwidth optimization must not remove this independence without adding an equally robust replacement. 3.4 Resource shadow copies -------------------------- The proxy keeps CPU shadow bytes for WC3 buffers and textures. Lock/Unlock and copy hooks update those shadows and increment a generation number when content changes. Vertex and index buffer updates are not normally transmitted as large generic UPDATE_BUFFER records. At each draw, the proxy selects only the byte range that the draw can reference. It then gives that slice a content identity and sends the slice, or a delta for the slice. This is a major reduction. A rotating dynamic ring buffer can be large, but a single draw may use only a small range. Texture updates are different. Texture content can be sampled by many draws, so the proxy sends reliable texture resource records and pixel blobs. It tracks format, mip level, rectangle, pitch, size, id, and generation. 4. THE W3CS WIRE UNITS ====================== 4.1 Browser envelope -------------------- Every WebRTC DataChannel message holds one complete W3CS envelope fragment. All numeric fields use little-endian byte order. Offset Size Meaning ------ ---- ------------------------------------------------------------- 0 4 ASCII magic "W3CS" 4 1 protocol version, currently 1 5 1 message kind 6 2 flags 8 4 session id 12 4 sequence number 16 4 source frame number, or zero for non-frame messages 20 2 fragment index 22 2 fragment count 24 4 fragment payload bytes 28 4 CRC-32 of the transmitted fragment payload Fixed envelope size: 32 bytes. Current message kinds: 1 HELLO 2 RESOURCE 3 FRAME 4 REPAIR 5 ACK 6 ERROR 7 RESOURCE_BUNDLE 8 RESOURCE_REFERENCE Current flags: bit 0, COMPRESSED bit 1, KEYFRAME/reliable recovery frame bit 2, LAST fragment bit 3, GEOMETRY_ANCHOR The native C++ relay currently refragments compressed browser messages at 16 KiB. The protocol permits a transmitted fragment payload up to 60 KiB. The smaller current value reduces the cost of abandoning one large SCTP user message after UDP loss. Resource and frame channels have independent sequence spaces. This is intentional. A dropped frame fragment must not appear to create a gap in the ordered resource sequence. 4.2 Record header ----------------- After message reassembly and decompression, a payload contains records. Each record begins with: uint8 opcode uint8 flags uint16 reserved; must be zero uint32 payload size Record header size: 8 bytes. The length allows a decoder to skip an unknown optional opcode. The current HELLO is only: {"producer":"wc3-d3d9-proxy","protocol":1} The older protocol plan says HELLO should advertise required opcodes. The current HELLO does not yet provide that list. Treat this as a compatibility gap, not as an implemented safety feature. 4.3 Frame resource dependency ----------------------------- The C++ relay prefixes every decoded FRAME record stream with: uint32 requiredReliableResourceSequence The browser does not apply that frame until it has processed the reliable resource channel through that sequence. A normal disposable frame waits up to 100 ms by default. A reliable recovery can wait much longer because its resource snapshot can be several MiB. This field prevents cross-DataChannel reordering from showing a draw before the texture or geometry base that it references. 5. CURRENT RECORD VOCABULARY ============================ The following list describes the current native and browser implementation. Sizes exclude the common 8-byte record header. Opcode 1: FRAME_STATE --------------------- Payload: 4 bytes. uint32 stateVersion, currently 1 This resets the browser's small per-frame D3D state. The rest of the snapshot records follow it. Opcode 2: CREATE_BUFFER ----------------------- Payload: 25 bytes. uint32 id uint32 generation uint32 byteSize uint32 usage uint32 formatOrFVF uint32 pool uint8 kind; 1=vertex buffer, 2=index buffer The browser keeps this descriptor. Draw bytes come from content blobs rather than from a complete browser-side emulation of Lock/Unlock. Opcode 3: UPDATE_BUFFER ----------------------- The native header reserves this opcode, but the current browser CommandState does not implement it. Current draw geometry is transmitted as content blobs. A new compressor must not start emitting opcode 3 without adding and testing its browser implementation. Opcode 4: CREATE_TEXTURE ------------------------ Payload: 32 bytes. uint32 id uint32 generation uint32 width uint32 height uint32 mipLevels uint32 usage uint32 D3D format uint32 pool Current supported formats are A8R8G8B8, X8R8G8B8, R5G6B5, X1R5G5B5, A1R5G5B5, A4R4G4B4, DXT1, DXT2, DXT3, DXT4, and DXT5. Opcode 5: UPDATE_TEXTURE ------------------------ Payload: 40-byte header followed by exact source bytes. uint32 id uint32 generation uint32 mipLevel uint32 x uint32 y uint32 width uint32 height uint32 pitch uint32 D3D format uint32 byteSize uint8 pixelOrBlockBytes[byteSize] The update can cover one region and one mip. DXT formats use block rows. Opcode 6: DESTROY_RESOURCE -------------------------- Current payload: 4-byte resource id. The browser removes any buffer and texture with that id and destroys the GPU texture if one exists. Opcode 7: DEFINE_BLOB --------------------- Payload: 24-byte identity followed by raw content. uint32 id uint32 reconstructedByteSize uint64 hashA uint64 hashB uint8 bytes[reconstructedByteSize] The id is content-derived. The recorder uses two table-CRC lanes plus the size, then resolves any live 32-bit id collision. It also compares bytes in the server cache before deduplicating. The browser reconstructs the bytes and verifies both 64-bit hash fields. These fields are efficient content-integrity and identity checks. They are not a cryptographic digest. Persistent resource bundles use SHA-256 separately. Opcode 8: DEFINE_BLOB_XOR ------------------------- Payload: 28-byte delta header plus one XOR byte for every reconstructed byte. blob identity as above uint32 baseBlobId uint8 xorBytes[reconstructedByteSize] The browser supports this form. The current recorder normally selects the more compact masked XOR form instead. Opcode 9: DEFINE_BLOB_XOR_MASK ------------------------------ Payload: 24-byte output blob identity uint32 baseBlobId byte-change mask only the non-zero XOR values For N reconstructed bytes: words = ceil(N / 4) maskBytes = ceil(words / 2), approximately N / 8 Each nibble describes which of four bytes in one 32-bit word changed. The values area contains one byte for every set mask bit. Example for a 32-byte slice: mask size: 4 bytes if only bytes 1, 9, and 28 changed: values size is 3 bytes encoded delta data before record/header overhead: 7 bytes raw slice: 32 bytes The browser copies the base blob, XORs only the selected bytes, and validates the reconstructed content hash. Opcode 10: RESET_BLOB_CACHE --------------------------- Payload: none. The browser clears decoded blobs. GPU vertex/index buffers derived from those blobs are also destroyed through the renderer reset callback. Opcode 11: DEFINE_BLOB_FLOAT16_DELTA ------------------------------------ Payload: 24-byte output blob identity uint32 baseBlobId uint32 vertexStride uint32 FVF float-word mask raw-byte mask binary16 delta values raw XOR values For N reconstructed bytes: words = ceil(N / 4) floatMaskBytes = ceil(words / 8) rawMaskBytes = ceil(words / 2) For eligible float32 words, the server tries: halfDelta = float16(currentFloat - baseFloat) reconstructed = baseFloat + float32(halfDelta) It accepts the half delta only when all values are finite and: abs(reconstructed - current) <= 0.0002 + abs(delta) * 0.0006 Position words are deliberately not eligible. They use bit-exact raw XOR bytes. Earlier half-position deltas moved projected edges by a fraction of a pixel and made static buildings appear to vibrate after browser upscaling. Eligible words include normals, point size, and texture coordinates when the FVF layout identifies them as float values. Diffuse and specular packed color words remain raw. Important: this format is bounded-lossy for eligible attributes. The output blob id and hash identify the reconstructed bytes, not the server's original float bytes. This makes later deltas and browser validation internally exact against the transmitted reconstruction. Opcode 12: UPDATE_TEXTURE_BLOB ------------------------------ Payload: 44 bytes. the same 40-byte UPDATE_TEXTURE header uint32 contentBlobId The pixel bytes live in a previously defined content blob. This avoids sending the same large texture data for each session-local texture identity. The browser pins a blob permanently after a texture references it. Geometry recovery pruning cannot then remove texture content. Opcodes 13 and 14: RESOURCE_SNAPSHOT_BEGIN and END -------------------------------------------------- Payload: none. These delimit a coherent recorder-side snapshot. The relay consumes the markers while assembling the reliable resource epoch. They are not part of normal browser CommandState interpretation. Opcode 16: SET_RENDER_STATE --------------------------- Payload: 8 bytes. uint32 D3D render-state id uint32 value Many examples used by the current renderer: 7 ZENABLE 14 ZWRITEENABLE 15 ALPHATESTENABLE 19 SRCBLEND 20 DESTBLEND 22 CULLMODE 23 ZFUNC 24 ALPHAREF 27 ALPHABLENDENABLE 28 FOGENABLE 34 FOGCOLOR 35 FOGTABLEMODE 36 FOGSTART 37 FOGEND 38 FOGDENSITY 137 LIGHTING 139 AMBIENT 141 COLORVERTEX 145 DIFFUSEMATERIALSOURCE 147 AMBIENTMATERIALSOURCE 148 EMISSIVEMATERIALSOURCE 174 SCISSORTESTENABLE The browser preserves all transmitted state ids in a Map. The WebGPU emulator currently interprets only the subset that its shader and pipeline use. Opcode 17: SET_TRANSFORM ------------------------ Payload: 68 bytes. uint32 transform state float32 matrix[16] Important state ids: 2 view 3 projection 16 texture transform for stage zero 256 world Opcode 18: SET_TEXTURE ---------------------- Payload: 12 bytes. uint32 stage uint32 texture id, or zero for none uint32 generation The browser records stage zero and stage one bindings. The current WebGPU shader samples only texture stage zero. Opcode 19: SET_FVF ------------------ Payload: 4-byte D3D flexible vertex format bitfield. The browser derives the vertex layout from position type, normal, diffuse, specular, and texture-count fields. Opcode 20: SET_STREAM_SOURCE -------------------------------- Payload: 20 bytes. uint32 stream uint32 buffer id uint32 generation uint32 byte offset uint32 stride The draw's transmitted blob already contains the referenced slice. The stride and FVF tell WebGPU how to interpret it. Opcode 21: SET_INDICES ---------------------- Payload: 8 bytes. uint32 index-buffer id uint32 generation The buffer descriptor tells the browser whether indices are uint16 or uint32. The draw uses a content blob for the actual index slice. Opcode 22: SET_MATERIAL ----------------------- Payload: 68 bytes, matching 17 float32 values. diffuse RGBA ambient RGBA specular RGBA emissive RGBA power Opcode 23: SET_LIGHT -------------------- Payload: 108 bytes. uint32 light index uint32 D3D light type 25 float32 values for colors, position, direction, range, falloff, attenuation, theta, and phi The browser stores the complete record. The current shader uses the first enabled directional light. Point/spot attenuation and full specular lighting are not implemented. Opcode 24: LIGHT_ENABLE ----------------------- Payload: 8 bytes: light index and Boolean enabled value. Opcode 25: SET_TEXTURE_STAGE_STATE ---------------------------------- Payload: 12 bytes: stage, state id, value. The current shader uses stage-zero texture-transform flags. It does not yet implement the complete D3D fixed-function texture-combiner algebra. Opcode 26: SET_SAMPLER_STATE ---------------------------- Payload: 12 bytes: sampler/stage, state id, value. Current interpreted examples for stage zero: 1 ADDRESSU 2 ADDRESSV 4 BORDERCOLOR 5 MAGFILTER 6 MINFILTER 7 MIPFILTER 8 MIPMAPLODBIAS Opcode 27: SET_VIEWPORT ----------------------- Payload: 24 bytes. uint32 x, y, width, height float32 minZ, maxZ Opcode 28: SET_SCISSOR ---------------------- Payload: 16 bytes: signed left, top, right, bottom. Opcode 29: CLEAR ---------------- Payload: 20 bytes. uint32 rectangle count uint32 flags uint32 ARGB color float32 depth uint32 stencil The current browser uses the color as the next WebGPU render-pass clear color. It clears depth to 1. It does not reproduce arbitrary Clear rectangles or all stencil semantics. Opcode 30: SET_CURSOR --------------------- Payload: 28-byte header plus BGRA pixels. uint32 hotX, hotY, width, height, pitch, D3D format, byteSize uint8 BGRA pixels[byteSize] Current limits are 256x256, A8R8G8B8, and width*4 pitch. The browser converts BGRA to RGBA and draws the actual WC3 cursor as a local overlay at display rate. Opcode 31: SHOW_CURSOR ---------------------- Payload: 4-byte Boolean visibility. Opcode 32: DRAW_PRIMITIVE ------------------------- Payload: 16 bytes. uint32 primitive type uint32 start vertex uint32 primitive count uint32 vertex content-blob id Opcode 33: DRAW_INDEXED_PRIMITIVE --------------------------------- Payload: 32 bytes. uint32 primitive type int32 base vertex uint32 minimum vertex uint32 number of vertices uint32 start index uint32 primitive count uint32 vertex content-blob id uint32 index content-blob id The recorder calculates the vertex and index byte ranges from these bounds and the active stride/index format before it creates the content blobs. Opcode 34: SET_WORLD_TRANSFORM_COMPACT -------------------------------------- Payload: 38 bytes instead of the normal 68-byte transform payload. The first 12 affine matrix components and matrix[15] use binary16. Translation matrix[12..14] stays float32. The proxy uses this form only if binary16 converts each compacted value back to exactly the original float32 value. Otherwise it sends opcode 17. Unlike opcode 11 geometry attributes, this transform optimization is lossless. 6. COMPRESSION IS A STACK, NOT ONE CODEC ======================================== It is useful to separate five reduction layers. A benchmark that measures only the final Zstandard ratio hides where the bytes came from. 6.1 Layer A: avoid recording redundant API work ----------------------------------------------- Current reductions: - sample at the selected 40 FPS capture rate; - do not emit repeated identical state setters; - maintain resource shadow copies instead of asking WineD3D to read back; - transmit only vertex and index ranges referenced by a draw; - use content ids so repeated geometry and texture bytes can be referenced; - emit the same blob only once per frame; - skip unchanged texture updates; - separate long-lived resources from disposable frames. These reductions are often more valuable than increasing the general-purpose compression level. 6.2 Layer B: content-addressed blob reuse ----------------------------------------- The proxy hashes each geometry or texture byte string and confirms matches with memcmp. The resulting 32-bit blob id is stable for identical content in the current live cache. Every definition includes the full two-part hash, and the browser validates the reconstructed bytes. Static meshes, repeated UI quads, repeated index patterns, and identical texture content can therefore be reused by id. For dynamic ring-buffer geometry, the proxy associates a delta base with a logical draw position, resource id, byte size, texture id, stride, and FVF. Because WC3 rotates through dynamic buffer offsets, this logical association is more useful than the physical byte offset alone. If draw ordinals shift because a unit or UI element appears, the proxy can search compatible slices with the same resource/layout/texture identity. This reduces the cost of creating a new full base. 6.3 Layer C: geometry-specific delta encodings ---------------------------------------------- The current preferred encodings are: - raw DEFINE_BLOB for a new or independently required base; - DEFINE_BLOB_XOR_MASK for sparse exact byte changes; - DEFINE_BLOB_FLOAT16_DELTA when eligible normals/UVs can use binary16; - exact raw bytes when the compact form would be larger or unsafe. Vertex positions remain bit-exact. Index data uses exact masked XOR deltas. World translation remains float32. A browser must validate the reconstructed content hash before it makes the blob available to a draw. 6.4 Layer D: Zstandard on each logical message ---------------------------------------------- The native C++ relay applies Zstandard after it has built the complete logical resource or frame message. Current levels: frame messages: Zstandard level 7 resource messages: Zstandard level 3 If compressed bytes are not smaller than plain bytes, the relay sends the plain message and clears the COMPRESSED flag. Compressed payload header without a dictionary: uint32 original decoded byte size uint8 codec=2 for Zstandard uint8 zero uint8 zero uint8 zero uint8 zstdFrame[] Header size: 8 bytes. Compressed payload header with a frame dictionary: uint32 original decoded byte size uint8 codec=3 for Zstandard-with-dictionary uint8 zero uint8 zero uint8 zero uint32 dictionary source frame number uint8 zstdFrame[] Header size: 12 bytes. The dictionary is an earlier complete decoded W3CS frame payload, including its four-byte resource dependency. Recovery frame: compressed independently, without a frame dictionary Geometry-anchor frame: compressed against the retained reliable recovery frame Following disposable frames: compressed against the latest geometry-anchor frame If the browser loses a dependent frame, later dependents can still decode because they do not use the lost dependent as their dictionary. If it loses an anchor, it drops only the dependent frames that name that anchor. A later anchor can resume from the retained recovery. The recorder currently marks a geometry anchor every four captured normal frames: one anchor plus three dependents. Some relay comments and the constant kCompressionAnchorInterval describe five. The actual dictionary decision follows the incoming GEOMETRY_ANCHOR flag. This comment/constant mismatch should be resolved before another engineer treats five as a wire guarantee. The browser performs Zstandard decode in a Web Worker with a pinned WASM build. It keeps decoded frame dictionaries by frame number, up to a bounded set. Raw DEFLATE codec id 1 remains supported by the JavaScript protocol code for older/reference traffic. The current native relay sends Zstandard. 6.5 Layer E: resource bundle cache ---------------------------------- The relay persistently identifies large immutable DEFINE_BLOB record streams. Current cache threshold: 16 KiB. Only a pure DEFINE_BLOB record stream is considered safe for persistent reuse. Create, bind, update, and destroy records contain session-local D3D ids or generations and are not placed in a cross-session bundle. First browser use: RESOURCE_BUNDLE payload = SHA-256 digest + decoded DEFINE_BLOB records The browser verifies SHA-256 after applying the live resource state, then saves the decoded record bytes in IndexedDB. Later browser use: RESOURCE_REFERENCE payload = SHA-256 digest + uint32 decoded byte size The browser retrieves the exact records from IndexedDB. If they are absent, it can fetch the retained bundle over the relay's HTTP recovery endpoint. Current browser cache bounds: approximately 192 MiB up to 4096 advertised digest keys The initial resource snapshot is still too large for production and has been measured above 40 MiB in this experiment. The planned production solution is a durable content-addressed WC3/map asset pack that a browser can fetch and cache before command mode claims the seat. 7. TRANSPORT AND LOSS BEHAVIOR ============================== Source: experiments/d3d9-command-stream/native/w3cs_webrtc_relay.cpp experiments/d3d9-command-stream/web/live-transport.js experiments/d3d9-command-stream/web/protocol.js 7.1 WebRTC channels ------------------- The C++ relay uses GStreamer's webrtcbin for ICE, DTLS, SCTP DataChannels, and Opus media transport. GStreamer does not compress or interpret D3D commands. Current DataChannels: resource ordered=true reliable carries descriptors, textures, immutable blobs, and other resource state recovery ordered=true reliable carries a complete geometry recovery frame frame ordered=false maxPacketLifeTime=250 ms carries disposable normal and geometry-anchor frames control ordered=true reliable carries input and control JSON An optional WebSocket frame plane exists as a diagnostic/fallback route. The browser can switch the active frame plane and then waits for a new recovery. 7.2 Why normal frames are not fully reliable --------------------------------------------- Retransmitting stale command frames can create visible delay. A normal frame is made disposable and independently reconstructible from a reliable geometry epoch/anchor. The relay can replace an unsent frame with a newer one. The application queue holds a pending frame and, at most, a latest replacement. It preserves an unsent compression anchor when a dependent would otherwise replace it. The SCTP buffered limit adapts between approximately 256 KiB and 1 MiB. A much larger buffer turns congestion into seconds of latency. A much smaller buffer can incorrectly classify one 55-90 KiB busy frame as congestion. Current wire fragment payload: 16 KiB. Every fragment has its own CRC-32. The browser renders a frame only after all fragments arrive and the whole message passes bounds and decompression checks. 7.3 Browser frame scheduling ----------------------------- The browser keeps a short reorder window. It uses the exact next frame-message sequence if available. After the reorder delay, it can skip a missing disposable message and select the newest independently decodable frame. The reorder delay adapts from observed RTT and is clamped to 20-80 ms. It does not wait one full RTT for a disposable frame. Recovery frames take priority. On recovery, the browser clears old geometry epoch pins and treats the new frame as the independent base. If a frame uses a Zstandard dictionary that the browser does not have, the browser drops the frame. It does not guess. If a geometry blob base is missing, it drops the frame and requests recovery. 8. BROWSER DECODING AND STATE RECONSTRUCTION ============================================ 8.1 Envelope validation ----------------------- The browser first checks: magic and protocol version known message kind exact payload length fragment coordinates CRC-32 fragment and reassembled-message limits Important current limits: fragment payload: 60 KiB maximum reassembled frame: 4 MiB reassembled resource message: 64 MiB pending fragmented messages: 128 records per message: 16,384 draws per frame: 4,096 one buffer descriptor: 64 MiB texture dimension: 8192 reconstructed texture storage: 256 MiB default blob store: 128 MiB The fragment reassembly key includes session, kind, frame, and first sequence. Conflicting duplicate fragments fail the message. 8.2 Decompression ----------------- DEFLATE uses the browser DecompressionStream API. Zstandard uses the pinned WASM decoder in a dedicated Worker. Before allocation, the browser reads the claimed decoded size and compares it with the kind-specific limit. For dictionary compression, the browser looks up the complete decoded base frame by the frame number in the 12-byte compression header. 8.3 Reliable resource ordering ------------------------------ Reliable messages are processed through a Promise chain. A frame cannot pass its four-byte required-resource-sequence barrier until that chain reaches the needed sequence. IndexedDB SHA-256 verification and persistence happen after the records become available to the live frame path. The cache write does not delay the current frame once transport integrity checks pass. 8.4 CommandState ---------------- CommandState turns the record sequence back into a D3D-like state machine. It maintains: buffer descriptors texture descriptors and mip bytes content blobs render-state Map transform Map texture-stage and sampler Maps texture bindings stream bindings index binding FVF material lights and enables viewport scissor clear color At each DRAW record, it copies the state that was active at that exact point. This is necessary because the browser decodes the complete frame first and submits it afterward. If draws held references to one mutable state Map, every draw would incorrectly use the final state of the frame. The draw object given to WebGPU contains the current state snapshots, texture ids/generations, reconstructed vertex bytes, and reconstructed index bytes. 8.5 Blob reconstruction ----------------------- The BlobStore performs raw, XOR, masked-XOR, and float16-delta reconstruction. It then calculates the same hash pair as the recorder. A mismatch rejects the blob. A delta with a missing base reports the base id and prevents the frame from rendering. Reliable texture blobs are permanently pinned. Recovery geometry is pinned for the active epoch. Disposable transient blobs are pruned after complete frames, except for the current draw set and active geometry anchor set. When decoded blob bytes reach a draw, WebGPU buffer caching uses: content blob id + GPU usage (VERTEX or INDEX) The bytes are uploaded once for that content id. Repeated draws reuse the GPU buffer. 9. WEBGPU RENDERING =================== Source: experiments/d3d9-command-stream/web/webgpu-renderer.js experiments/d3d9-command-stream/web/texture-codec.js 9.1 Output target ----------------- The accepted lab source is 1600x1200, exact 4:3. The browser canvas uses the source aspect ratio and creates a physical-pixel WebGPU target based on the element width and devicePixelRatio. Current lab settings can use 1.25x client supersampling with a 2560-pixel maximum output width. A 1600x800 or 1600x900 page viewport letterboxes the 4:3 game. Fullscreen centers the same 4:3 surface. It must not stretch it to 16:9. The game creates its own UI/text geometry at the 1600x1200 source resolution. WebGPU upscaling cannot create font detail that WC3 did not generate. 9.2 Vertex layout from FVF -------------------------- The renderer maps the active D3D FVF to WebGPU attributes: location 0: position, float32x3 or XYZRHW float32x4 location 1: normal, float32x3 when present location 2: diffuse, BGRA unorm8x4 when present location 3: first texture coordinates, float32x2 when present location 4: specular, BGRA unorm8x4 when present XYZRHW vertices are interpreted as screen-space UI. The shader maps their pixel x/y directly to normalized device coordinates. World vertices use: world * view * projection The matrices are placed in one dynamic uniform slot per draw. 9.3 One batched uniform upload ------------------------------ Each draw gets a 512-byte aligned uniform slot. The live WGSL data currently uses 92 float32 values for: model-view-projection matrix world matrix texture-transform rows light direction and color material diffuse, ambient, and emissive feature flags source dimensions alpha-test reference material color sources sampler border color and LOD bias fog color and parameters The renderer prepares all slots in one Float32Array and makes one queue.writeBuffer call before drawing the frame. 9.4 Pipeline cache ------------------ A WebGPU pipeline is cached by state that changes the pipeline definition, including: FVF stride alpha blending enable and factors depth test depth write depth compare function primitive topology index format orthographic/UI classification cull mode Per-draw values that do not require another pipeline are uniforms. 9.5 Fixed-function shader behavior ---------------------------------- Current vertex behavior: - screen-space XYZRHW mapping for Classic UI; - world/view/projection transform for game geometry; - optional directional-light diffuse + ambient + emissive calculation; - optional vertex diffuse/specular material sources; - stage-zero texture-coordinate transform; - pass color, UV, and fog depth to the fragment shader. Current fragment behavior: - sample texture stage zero; - multiply sampled color by vertex/material color; - emulate border color when U/V uses D3DTADDRESS_BORDER; - alpha-test discard using the captured reference; - apply supported linear fog; - return color to WebGPU blending. This is a targeted fixed-function emulator. It does not execute D3D shader bytecode. 9.6 Textures ------------ For BC/DXT textures: - if the WebGPU device supports texture-compression-bc, the browser uploads WC3's DXT bytes directly as BC1, BC2, or BC3; - otherwise JavaScript decodes DXT1-5 to RGBA8 and uploads that. For 32-bit and 16-bit uncompressed D3D formats, JavaScript converts pixels to RGBA8 before upload. Mips are uploaded into one WebGPU texture. The sampler cache uses captured address, magnification, minification, mip, and LOD-bias state. Classic UI and text use linear magnification in the current lab, even if the world sampler requested point magnification. This reduces harsh stair steps on modern high-DPI screens. World geometry keeps the captured filter choice. If a texture is absent, the current renderer uses a one-pixel white texture. If a GPU texture is older than a referenced generation, it currently retains the last valid GPU copy and records a stale-texture diagnostic instead of dropping the complete frame. This policy can produce a temporary wrong or white surface. A compression redesign must not confuse that existing fallback with correct texture delivery. 9.7 Drawing primitives ----------------------- Current WebGPU rendering covers the WC3 triangle paths: D3DPT_TRIANGLELIST D3DPT_TRIANGLESTRIP D3DPT_TRIANGLEFAN WebGPU has no triangle-fan topology. The browser creates and caches a generated uint16 fan index buffer. Point and line primitive records can be captured, but the current renderer does not submit them. This is part of the targeted-WC3 subset. Each draw applies the captured scissor and viewport. UI/RHW uses the complete canvas viewport. World viewports scale from source pixels to WebGPU target pixels. All commands are encoded into one render pass. The browser then submits one finished WebGPU command buffer for the frame. 9.8 Cursor ---------- The WC3 cursor bitmap comes from SetCursorProperties as a reliable resource. The browser draws it as a separate local overlay at browser display rate. This makes cursor movement responsive without waiting for the next 40 FPS command frame. The renderer can suppress draw calls that use the captured server cursor atlas texture. The server cursor is also parked outside the game window after input motion. These controls prevent the delayed cyan server reticle from appearing under the local WC3 hand. 10. DIVERSE END-TO-END EXAMPLES =============================== 10.1 A normal textured Orc building ----------------------------------- The game may issue, in simplified order: SetTransform(WORLD, building transform) SetTransform(VIEW, camera) SetTransform(PROJECTION, perspective) SetTexture(0, Orc building texture) SetFVF(XYZ | NORMAL | DIFFUSE | TEX1) SetStreamSource(vertex buffer, stride) SetIndices(index buffer) SetRenderState(ZENABLE, TRUE) SetRenderState(ZWRITEENABLE, TRUE) SetRenderState(CULLMODE, CCW) DrawIndexedPrimitive(TRIANGLELIST, ...) The recorder: 1. suppresses setters identical to the state already active; 2. encodes the world transform as 38 bytes only if the half components are exact; 3. calculates the vertex and index slices referenced by this draw; 4. reuses content ids, or creates masked/float16 deltas from safe bases; 5. writes opcode 33 with the two resulting blob ids. The browser: 1. reconstructs and hashes the geometry blobs; 2. snapshots the active state at opcode 33; 3. creates/reuses WebGPU vertex and index buffers; 4. selects a cached perspective, depth-writing, culled pipeline; 5. binds the texture and sampler; 6. runs directional fixed-function lighting in WGSL; 7. submits the indexed triangles. 10.2 An animated tree or CPU-skinned unit ----------------------------------------- WC3 rewrites dynamic vertices each frame. Positions can change and alpha-tested leaf geometry can overlap. The recorder keeps a reliable or recent-anchor base for the logical draw. It uses: exact XOR-mask bytes for position changes; binary16 deltas for eligible normals/UV values within tolerance; exact draw order, alpha-test state, depth-write state, and cull mode. This exact position rule is important. A previous all-float16 experiment made buildings and trees vibrate. Another previous pose-reuse experiment made trees freeze for several frames and then pop. Both are examples of bandwidth savings that are not visually acceptable. 10.3 Chat text or a top-bar UI glyph ------------------------------------ WC3 submits screen-space XYZRHW quads with an atlas texture, diffuse color, and alpha blending. The command stream contains: FVF with XYZRHW, diffuse, and texture coordinates; UI atlas texture binding; sampler state; alpha blending state; small repeated quad/index blobs; one or more draw records. The browser classifies the FVF as RHW, bypasses world/view/projection, maps source pixel coordinates to clip space, disables depth for the UI pipeline, uses linear magnification, and blends the glyph texture. The browser does not replace WC3's font with HTML text. Therefore the glyph shape and source detail remain the Classic bitmap asset generated by WC3. 10.4 Alpha-tested tree leaves ----------------------------- The important captured states include: ALPHATESTENABLE ALPHAREF ZENABLE ZWRITEENABLE CULLMODE texture and sampler address modes The fragment shader samples stage zero and discards pixels at or below the captured alpha reference. The pipeline can still write depth while alpha blending is active, matching a WC3 pattern that earlier emulation got wrong. 10.5 Gold mine with lighting and texture wrapping ------------------------------------------------- The gold mine uses perspective geometry, normals, material, lighting, texture, and sampler state. The browser reconstructs the world transform and first enabled directional light, then calculates ambient/emissive/diffuse lighting. If the texture update or generation is missing, the current white/stale-texture fallback can make the mine look white or incomplete even when its geometry is correct. This is why resource correctness must be measured separately from geometry compression. 10.6 Minimap or another changing UI texture -------------------------------------------- The game locks or copies a texture region. The proxy updates its texture shadow and increments the texture generation. At a draw boundary, dirty sampled textures are recorded before the draw. The reliable path sends the pixel content as a DEFINE_BLOB plus an UPDATE_TEXTURE_BLOB reference where possible. The frame names the required resource sequence. The browser waits for that sequence, reconstructs the blob, updates the correct mip/rectangle, and calls queue.writeTexture. This data must not move to the expiring frame channel unless the protocol adds a safe independent texture-generation recovery model. 10.7 D3D triangle fan --------------------- The recorder sends primitive type 6 plus the vertex slice and primitive count. The browser creates indices: 0,1,2, 0,2,3, 0,3,4, ... It caches that index buffer by fan size and submits drawIndexed in WebGPU. 10.8 Clear, viewport, and scissor --------------------------------- Clear records update the frame's clear color. Viewport and scissor changes are retained in the state snapshot and copied per draw. The browser starts one WebGPU render pass with color/depth clear, then scales source scissor and viewport coordinates to the physical-pixel canvas. 10.9 Actual WC3 cursor ---------------------- SetCursorProperties sends the BGRA cursor image and hot point on the reliable resource plane. Browser code converts it to RGBA. Pointer movement updates the overlay immediately without waiting for game-render cadence. The corresponding input still goes to the server so WC3 hover, selection, and camera behavior remain authoritative. 10.10 A lost disposable frame ----------------------------- Suppose the browser has: reliable recovery R geometry/compression anchor A dependent frames B, C, D If B is lost, C and D still use A and can decode. If A is lost, B/C/D name a missing Zstandard dictionary and are dropped. The next anchor is compressed against R and can resume without waiting for a new round-trip recovery. No dependent frame may use the immediately previous disposable dependent as its only base. That would turn one packet loss into a chain failure. 10.11 A persistent resource-cache hit ------------------------------------- On first use, the browser receives: SHA-256 + raw DEFINE_BLOB records It verifies and stores the records. On the next compatible session, it receives only: the same 32-byte SHA-256 + 4-byte decoded size It reads the exact record stream from IndexedDB. Session-local CREATE_TEXTURE and binding records still arrive normally. 11. CURRENT PERFORMANCE OBSERVATIONS ==================================== Do not mix measurements from different experimental profiles. Earlier 1024x768 XOR-mask bundle: approximately 10.7 KiB compressed per steady frame approximately 3.4 Mbit/s at 40 FPS Current native 1600x1200 command lab: approximately 460 draws in a representative scene frame capture cap: 40 FPS game cap: 40 FPS measured Warcraft + recorder: approximately 0.126 vCPU in one sample measured C++ WebRTC + Opus relay: approximately 0.091 vCPU in one sample combined before small Wine helper costs: approximately 0.217 vCPU direct UDP browser sample frame gap median: 24.6 ms direct UDP browser sample frame gap p95: 50.7 ms The browser state-build plus GPU-submit measurement has been approximately 2-4 ms for the modular live renderer. An earlier saved-bundle proof measured approximately 0.9 ms of browser render work. Current C++ comments report busy late-game encoded frames commonly around 55-90 KiB. At 40 FPS that is above the desired production bandwidth gate. The value varies substantially by scene. It must be measured by frame type and opcode contribution, not treated as one fixed bitrate. The desired release gate in README is: median below 5 Mbit/s p95 below 8 Mbit/s at 40 FPS across the validation corpus The current busy-scene path has not met that gate consistently. The initial resource bootstrap also remains too large. Earlier runs exceeded 40 MiB before persistent pack caching. 12. WHAT IS LOSSLESS, BOUNDED-LOSSY, OR APPROXIMATE =================================================== Wire/capture behavior that is intended to be exact: record framing and integer fields vertex position bytes index bytes sparse raw XOR changes world translation compact world transform only when binary16 round-trips exactly texture bytes as captured from WC3 resource SHA-256 bundle identity per-fragment CRC-32 reconstructed blob hash verification draw order Bounded-lossy behavior: binary16 deltas for eligible normals, UVs, and point-size float words accepted only within the explicit numeric tolerance future deltas use the reconstructed wire bytes as their identity/base Rendering approximations in the current fixed-function emulator: only stage-zero texture sampling is used by the shader generic D3D texture combiners are not fully implemented only the first enabled directional light is used point/spot lighting and complete specular behavior are not implemented supported linear fog subset only arbitrary Clear rectangles/stencil behavior not reproduced point and line primitives not submitted stale or absent textures can use an old GPU copy or white texture UI magnification is deliberately forced to linear Compression work should not be blamed for all visual differences. A side-by- side failure must first be classified as missing data, wrong reconstruction, wrong D3D state interpretation, or an intentional renderer approximation. 13. CURRENT FAILURE AND RECOVERY RULES ====================================== A browser frame is rejected when, among other cases: envelope identity or CRC is wrong; a fragment set is incomplete or conflicting; claimed sizes exceed limits; Zstandard dictionary is absent; decompressed size differs from the claimed size; resource dependency does not arrive in time; a blob delta base is missing; reconstructed blob hash does not match; required vertex/index bytes are missing; draw count or resource bounds exceed limits. On a disposable frame failure, the browser waits for an independent recovery path. It must not apply part of the frame. The renderer raises commandReady only after a complete frame has reached WebGPU submission. The production design must keep H.264 video active until that edge, and must be able to return to video after command-mode failure. Safe mid-session WineD3D raster-state restoration is not complete, so this fallback is still a release blocker. 14. HIGH-VALUE COMPRESSION REVIEW TARGETS ========================================= These are review areas, not instructions to change correctness rules. 14.1 Measure bytes by semantic source first ------------------------------------------- The C++ relay already accumulates per-opcode byte counts and counts for frame and resource records. A serious review should export, per scene and frame type: plain bytes by opcode bytes after blob delta coding but before Zstandard Zstandard bytes for recovery, anchor, and dependent frames geometry bytes by FVF, stride, draw slot, and texture id raw versus XOR-mask versus float16-delta selection mask bytes versus changed-value bytes state snapshot bytes versus draw/blob bytes static resource bootstrap bytes by texture/map/archive identity p50, p95, p99, and maximum frame sizes wire bytes including envelope/fragment overhead encoder CPU per frame and per byte saved Without this profile, a new codec can optimize a small record family while the dynamic vertex slices still dominate. 14.2 Move immutable assets out of the live session -------------------------------------------------- The largest straightforward win is likely the planned durable content pack: stable WC3 textures stable model geometry map assets other patch/profile-specific immutable blobs The browser should fetch by cryptographic content id, cache once, and prove the exact pack/profile identity before command mode begins. The live reliable channel should send only session-local descriptors and truly dynamic content. Do not persist arbitrary batches containing session-local D3D ids. 14.3 Encode the complete frame-state snapshot more structurally --------------------------------------------------------------- The complete state snapshot is necessary for loss independence, but its byte representation can be improved. Possible safe directions: canonical state order with a presence bitmap; field-specific packed values instead of an 8-byte record header per setter; a versioned compact snapshot opcode; a reliable state epoch plus a complete small delta snapshot that can be reconstructed without any disposable predecessor; common snapshot templates identified by content hash. Any design must preserve the exact state active before the first draw and all in-frame changes between draws. 14.4 Improve geometry predictors without chaining frame loss ------------------------------------------------------------- Possible predictors to test: previous reliable base for the same logical draw slot; latest reliable/recoverable geometry anchor; rigid-mesh base plus a transmitted transform, when proven equivalent; vertex-lane predictors for normals and UVs; run-length or entropy coding of sparse byte masks; index-pattern dictionaries; interleaved attribute separation before entropy coding; mesh/animation-aware coding only for a proven WC3 FVF subset. Non-negotiable rule: A disposable frame must not require an arbitrary earlier disposable frame that the browser may have lost. Position quantization needs exact image-difference tests. The existing building vibration is proof that a numerically small error can be visually large. 14.5 Revisit mask representation -------------------------------- The current XOR mask reserves one bit per byte, packed as one nibble per 32-bit word. For different change densities, alternatives may be smaller: sparse changed-byte indices with delta-coded offsets; run-length spans; word-level bitmap followed by per-word byte mask; SIMD-friendly bitset plus values; choose among modes per blob using exact encoded-size comparison. The decoder must remain bounded and validate that it consumes exactly all mask and value bytes. 14.6 Revisit frame-level dictionaries -------------------------------------- The current Zstandard dictionary is a complete prior decoded frame payload. Useful experiments include: trained static dictionaries by WC3 patch/FVF/scene family; separate dictionaries for state, geometry definitions, and draws; smaller selected dictionary material instead of a whole frame; deterministic anchor cadence based on encoded cost and loss budget; parallel Zstandard contexts with no allocation per frame; dictionary ids tied to reliable content hashes rather than only frame ids. Measure both bytes and encode/decode time. A better ratio that increases server CPU above the density target is not automatically useful. 14.7 Pack fixed-width record fields carefully --------------------------------------------- Many values are small but use uint32: state ids stage ids primitive types counts FVF/layout classes common texture dimensions draw-local blob references A version-2 compact record stream could use canonical varints, local id tables, or frame dictionaries. However, random access and bounds must stay simple. Zstandard already compresses repeated zeros and small values well, so measure the net gain after Zstandard before adding decoder complexity. 14.8 Avoid redundant geometry identities in one frame ----------------------------------------------------- The recorder already emits a content blob only once per frame. Further review can check whether multiple draw records can use: a frame-local blob-id table; a frame-local texture/state table; repeated draw templates plus compact changed fields; repeated index patterns shared across UI quads and model parts. Do not reorder transparent draws to improve compression. D3D draw order is part of the image. 14.9 Separate dynamic textures from static textures --------------------------------------------------- Profile UPDATE_TEXTURE and UPDATE_TEXTURE_BLOB by id, dimensions, frequency, and changed rectangle. Possible improvements: immutable texture pack for static assets; tile or block-level hashes for large dynamic textures; reliable changed-block deltas against an acknowledged texture generation; merge repeated updates before the next sampled frame; avoid retransmitting unused mip levels after proving the game's sampling. Texture deltas require an acknowledged base generation. A lost reliable base must never produce persistent white or corrupted materials. 14.10 Keep network framing independent from graphics coding ----------------------------------------------------------- Application fragments, SCTP messages, and D3D records solve different problems. A graphics codec should output a bounded logical frame. The transport can then choose fragment size and partial reliability from measured network behavior. Current 16 KiB fragments are a transport choice. They are not a geometry compression unit. 14.11 Decoder and upload cost matters ------------------------------------- Measure in the browser: Zstandard Worker time and queue depth blob reconstruction time by opcode hash verification time allocations and copies GPU buffer creation versus reuse texture decode/upload time CommandState cloning cost per draw uniform build time WebGPU encode and submit time Potential implementation wins include WASM SIMD for masks/hashes, pooled buffers, transferable ArrayBuffers, stable GPU ring buffers, and fewer Map clones. These can improve smoothness even when wire bytes do not change. 15. CORRECTNESS INVARIANTS FOR ANY NEW COMPRESSOR ================================================= The following rules should be treated as protocol requirements: 1. Never block the Warcraft render thread on compression or network I/O. 2. Queue a fragmented resource or frame completely, or not at all. 3. Never render a partial frame. 4. Every normal frame must be recoverable without an arbitrary lost normal predecessor. 5. A frame must declare the reliable resource sequence it needs. 6. Resource ids and generations must not silently cross sessions. 7. Content-addressed bytes must be verified after reconstruction. 8. Dynamic texture deltas must name a known, exact generation/base. 9. Keep vertex positions exact unless image tests prove a new representation has no visible movement across the full corpus. 10. Preserve D3D draw order, state changes between draws, alpha/depth behavior, viewport, and scissor. 11. Bound every count, size, dictionary, cache, and allocation before use. 12. A decoder error must fail command mode and retain the existing video fallback. It must not leave a corrupted interactive session. 13. A new protocol or opcode needs an explicit version/feature handshake. 14. Do not use a better compression ratio as the only acceptance test. Measure encoder CPU, decoder CPU, browser frame cadence, recovery time, and images. 16. RECOMMENDED VALIDATION CORPUS FOR COMPRESSION CHANGES ========================================================= At minimum, test: loading screen and UI text; early game with mostly static bases; large late-game fight with many CPU-skinned units; Night Elf trees and alpha-tested leaves; Orc, Human, Undead, and Night Elf buildings; gold mines and neutral buildings; rain, fog, particle-heavy scenes, and shadows; minimap and dynamic UI textures; chat, observer UI, replay controls, menus blocked by the product UI; camera pan, minimap click, selection, pause, and replay-speed changes; different maps and all supported Classic patch families; source 1600x1200 and physical-pixel browser upscale; normal window, high-DPI display, and centered 4:3 fullscreen. Network tests: stable 30 ms RTT; stable 100 ms RTT; variable 30-100 ms RTT; controlled loss and reordering; one lost dependent frame; one lost geometry anchor; resource channel delayed behind frame channel; reconnect and complete resource bootstrap; WebGPU device loss and video fallback. For every case, collect: exact server reference image; browser image at the same authoritative frame/camera; pixel difference and a human visual check; plain and encoded bytes by opcode; wire p50/p95/p99; server capture and codec CPU; browser decode and submit time; frame gaps and dropped-frame reason counts. 17. IMPORTANT CURRENT GAPS AND DOCUMENTATION WARNINGS ==================================================== 1. The command path is still a lab. Production still uses H.264 fallback. 2. The current initial resource bootstrap is too large. Durable pack caching is incomplete. 3. Busy frames can be 55-90 KiB after current coding and Zstandard. The desired production bitrate gate is not consistently met. 4. The current fixed-function renderer implements the WC3 subset observed so far, not all D3D9. Visual parity must be proven on a wider corpus. 5. Texture stage one is stored but not sampled by the WGSL shader. 6. General D3D texture combiners, point/spot lights, full specular, point/line primitives, stencil, and arbitrary Clear rectangles are incomplete. 7. Missing textures can currently render white, and a newer requested generation can temporarily use the last GPU texture. Treat this as a known artifact policy, not successful delivery. 8. The HELLO message does not yet advertise required opcode capabilities. 9. Opcode 3 exists in the native enum but is not implemented by CommandState. 10. Source comments disagree on a four-frame versus five-frame anchor interval. The recorder's current GEOMETRY_GOP=4 and emitted flag control reality. 11. The older WIRE-PROTOCOL.md table does not list newer message kinds 7/8 or all newer opcodes. w3cs_protocol.h plus browser code is the current source. 12. Safe mid-session switch back from no-raster command mode to WineD3D video is not complete. 13. Command mode currently targets WC3 Classic D3D9. The D3D11 -> DXBC -> WGSL concept is a separate future design and is not implemented here. 18. SOURCE MAP ============== Recorder and semantic geometry codec: experiments/d3d9-command-stream/native/d3d9_proxy.c Shared native protocol definitions: experiments/d3d9-command-stream/native/w3cs_protocol.h Native Zstandard, batching, WebRTC channels, audio, input, and backpressure: experiments/d3d9-command-stream/native/w3cs_webrtc_relay.cpp Browser envelope parser, CRC, fragmentation, and decompression: experiments/d3d9-command-stream/web/protocol.js experiments/d3d9-command-stream/web/zstd-decoder.js experiments/d3d9-command-stream/web/zstd-worker.js Browser reliable/disposable scheduling and recovery: experiments/d3d9-command-stream/web/live-transport.js Browser D3D state and record interpretation: experiments/d3d9-command-stream/web/command-state.js experiments/d3d9-command-stream/web/blob-store.js Browser texture conversion and BC upload choice: experiments/d3d9-command-stream/web/texture-codec.js Browser WebGPU fixed-function renderer: experiments/d3d9-command-stream/web/webgpu-renderer.js Browser content-addressed persistent resource cache: experiments/d3d9-command-stream/web/resource-bundle-cache.js End-to-end lab page and session wiring: experiments/d3d9-command-stream/poc/interactive-live.html experiments/d3d9-command-stream/poc/run-e2e-lab.sh Protocol overview and release gates: experiments/d3d9-command-stream/README.md experiments/d3d9-command-stream/WIRE-PROTOCOL.md 19. SHORT REVIEW SUMMARY ======================== The browser receives a versioned stream of resource, state, geometry, draw, cursor, and frame-boundary information. It does not receive pixels for every frame and it does not remotely execute synchronous D3D calls. The server currently reduces bandwidth in this order: suppress redundant D3D state copy only draw-referenced geometry slices content-deduplicate blobs encode exact XOR masks and bounded float16 attribute deltas reuse reliable recovery/anchor bases Zstandard-compress each complete message reuse immutable resource blobs through SHA-256 browser caching drop stale disposable frames instead of building latency The browser reverses those operations, validates every boundary and content identity, reconstructs per-draw D3D state, caches GPU resources, emulates the observed fixed-function pipeline in WGSL, and submits the frame through WebGPU. The best near-term bandwidth work is likely: 1. profile current bytes by opcode and geometry layout; 2. remove immutable WC3/map assets from live bootstrap through a durable content pack; 3. improve dynamic geometry prediction and sparse-mask coding while keeping position bytes and loss independence safe; 4. compact the complete small-state snapshot; 5. tune or train frame dictionaries with measured CPU and recovery costs. Any successful change must preserve the recovery graph. A smaller frame that depends on a packet the browser may have lost is not a real improvement.