Built for hordes

From docs/performance.md, Measuring

Measure before and after any change that touches a hot path:

cargo run --release -p sim --example bench --features profile -- 2000 600 passive
cargo run --release -p sim --example bench --features profile -- 10000 900
cargo run --release -p emberdeep -- --bench 1500 --bench-secs 20
cargo run --release -p net --example bandwidth -- 2000 600 passive [players=4] [loss=0.05 snap-delay=3 ack-delay=3]
cargo run --release -p bots -- --bots 4 --secs 20 --horde 2000 --god --passive [--lag-ms 50 --loss 0.05]

The first two print per-tick cost with a per-phase breakdown. The client bench disables vsync, keeps the window on top (macOS throttles occluded windows), forces continuous updates and logs FPS, frame time and net stats. bandwidth measures snapshot bytes and encode/apply cost on a simulated lossy network and checks the mirror against the sim every tick. emberdeep-bots runs a real server with N bot clients and reports server tick cost (sim / encode / net), bandwidth, RTT and consistency. EMBERDEEP_MONSTER_GLB=path.glb gives every monster kind that mesh in the client, for vertex-budget runs (design/perf/vertex-budget/run.sh).

Budgets: sim tick < 2 ms at 2,000 awake monsters; server tick (sim + encode for 4 clients) < 4 ms; per client < 64 KB/s typical, < 150 KB/s worst case.

The tight-loop sim bench understates real server cost by ~3.5x. The same tick costs 0.45 ms back-to-back but 1.6 ms when paced at 30 Hz with sleeps in between (the M2 drops clocks while idle; a QoS hint didn't help). On a busy machine, e.g. with clients rendering, it is back near the tight numbers. Judge server budgets with emberdeep-bots, judge sim changes with bench.

From docs/performance.md, Baselines (M2 Pro, 2026-10-23; sim rows re-measured after the N-player refactor, which changed the seed-42 map)

Skinned, animated characters (spike, 2026-10-25)

The client draws no skeletal animation yet. --skinned N (crates/client/src/skinned.rs) spawns N copies of a rigged concept model through Bevy's own glTF loader, out of step with each other, without shadows, to price it: --bench 0 --bench-secs 20 --offscreen --skinned 1000 [--skinned-still] [--skinned-model PATH]. Sereth (Tripo, 65-bone Mixamo skeleton, 2K PBR textures; a 4.9k-vertex variant decimated in Blender), 1,000 copies, 1600x900, load 8-9, baseline 4.7-5.0 ms (201-214 FPS):

• Playing clips is nearly free; skinning is not. Walking and standing still cost the same, so sampling the animation isn't the cost. A skin costs about twice the same mesh without one, and vertex count dominates (25k vertices skinned is ~5x the 4.9k one).
• Heroes and NPCs can be skinned as they are: a handful of copies is well under 1 ms even at 25k vertices.
• Hordes can't, as built: ~8 µs per skinned 4.9k-vertex copy is ~17 ms at 2,000 monsters, the whole frame budget. Hordes need something cheaper: vertex animation baked into textures (instancing-friendly, read in the vertex shader), fewer bones, or skinning only near the camera.
• Even unskinned, a textured copy here costs far more than a vertex-coloured monster (2,000 monsters at 21k vertices: 8.4 ms in the table above). Not split yet between the textures, the per-copy entity hierarchy and batching. macOS reports no GPU pass timings, so CPU and GPU shares are unknown too (Xcode's Metal frame capture would tell).

GPU frame (M2 Pro, 2026-10-25): the 120 Hz target

Alex's display is 120 Hz (ProMotion) and the window is 3200x1800 pixels. Under vsync a frame over 8.3 ms shows at 60, so the GPU frame needs to stay under ~7.5 ms. The game is fill-bound at Retina: cost scales with pixels, not with monsters.

Measuring it. Compare builds with --offscreen-size 3200x1800 and interleaved rounds:

python3 design/perf/gpu_ab.py target/release/emberdeep 3 8 'a||--start cinder-galleries --offscreen-size 3200x1800' 'b|BIN=/path/to/other|--start cinder-galleries --offscreen-size 3200x1800'

Each config is label|ENV=v BIN=path|args; the script prints median fps and ms. Every run opens an always-on-top window and saturates the GPU, so keep batches small: Alex works on this machine at the same time. Pitfalls:

• A windowed bench on a locked screen reports ~250-370 fps for anything (macOS skips the frames). Offscreen images still render.
• AutoNoVsync (the bench's mode) caps a windowed, near-empty frame at ~80-90 fps on macOS; vsync reaches 115-120. Don't read a windowed bench's fps as the game's.
• Background load shifts results by 10-20%. Interleave A and B in the same run; never compare against a number from another day.

What a 3200x1800 frame of the Cinder start costs (frame 19.4 ms after the two fixes below; savings when each piece is turned off): stone floor and walls 9-11 ms, post-processing and the HUD together 6.9 ms (SSAO 3.0, bloom 2.0, HUD 1.2, SMAA 0.7, tonemapping 0.6, grade 0.4), shadows 1.7, environment map 1.3. In the stone shader, the textures, voronoi cracks and anisotropy explain only ~3 ms; the rest is the lit material over the whole screen.

Load average was 6-9 during these runs.

Netcode baselines (M2 Pro, 2026-10-24, release; load average 3-9 from other work, so p99s are noisy)

Lessons learned: separation was 85-98% of sim time until the cell cap and cell-ordered data; per-model vertex count matters mostly through shadows (the main pass takes ~4k vertices per monster at 2,000 for free; see the vertex-budget row); point-light shadows render six passes, so the player's shadow caster is a spotlight. Metal doesn't report per-pass GPU timings through RenderDiagnosticsPlugin, and under heavy load its counter buffers fail to allocate and the GPU device is lost, so the bench doesn't add it on macOS: use Xcode's Metal frame capture for GPU attribution.