007 — Designing Shadows Before Writing Them
No screenshot this time: the decisions behind ARAL's shadow maps, and why the static / dynamic command buffer split was dropped
2026-11-09 → 2026-11-11
No image in this post. Shadows are the most expensive thing ARAL will have done so far — on the Steam Deck, the usual share for shadows in a 60 fps game is 1.5 to 3 ms out of 16.6 — and they touch everything: culling, command recording, the frame UBO, the PBR shader, the toon mode, the sky. So before writing a line of the renderer side, a few days went into deciding what to build. This post is that discussion.
Along the way, one architecture decision taken months ago fell: the split between a pre-recorded "world" command buffer and a per-frame "dynamic" one.
The frame budget, first
Every step from now on states its line in the budget and is judged against it. Deck, 60 fps = 16.6 ms:
| GPU | Usual share | ARAL target |
|---|---|---|
| Opaque scene (lights and shadow reads included) | 3–5 ms | ≤ 4 ms |
| Shadows | 1.5–3 ms | ≤ 3 ms (3 cascades ~1–2 ms + local lights) |
| Lighting, sky, fog | 1–3 ms | ≤ 1.5 ms |
| Transparent, VFX, particles | 1–2 ms | ≤ 2 ms |
| Post: bloom, AA, tonemap | 1–2 ms | ≤ 2 ms |
| UI | ~0.3 ms | ≤ 0.3 ms |
| Margin | ~2 ms | ≥ 2 ms |
The "usual share" is an order of magnitude from GDC talks, not a measurement of a specific game. The ARAL targets are estimates. A third column, "real Deck", only gets numbers measured on the Deck — a desktop number multiplied by the ×12 GPU factor from devlog 004 stays an estimate.
Dropping the static / dynamic split
The original plan, decided long ago: record the static world once in a cmd_world command buffer,
and only re-record the moving objects every frame in cmd_dynamic. Almost no CPU for the scenery. It looked
obviously right.
Then shadows forced a real count. Reference zone: 2000 objects / ~2 M triangles, 500 visible, the sun with 3 cascades, 2 point lights near the player. All numbers are Deck estimates, not measured.
A. No split (what ARAL does today). Culling, sorting and recording on the CPU every frame.
| Item | Draws | CPU |
|---|---|---|
| Culling + sort, all views | 2000 objects × 4 | ~0.1 ms |
| Player view | 500 | 0.15–0.5 ms |
| 3 cascades | ~1200 | 0.35–1.2 ms |
| Lights near the player | ~300 | 0.1–0.3 ms |
| Total | ~2000 | ~0.7–2.1 ms |
On the GPU, only visible objects are drawn, sorted front to back, with early-Z.
B. Split, cmd_world pre-recorded without culling (the old plan).
CPU: almost 0 for the static part, ~0.05 ms left for the moving objects.
GPU: the whole scenery is drawn in every view and every cascade — a pre-recorded buffer can't be culled per frame.
2 M triangles × (1 view + 3 cascades) = 8 M triangles a frame. At 0.5–1 billion triangles per second in practice
on the Deck: +8 to 16 ms of GPU. We save 2 ms of CPU at best and lose 8 to 16 of GPU. Excluded.
C. GPU-driven: compute culling + indirect draws (vkCmdDrawIndexedIndirectCount).
CPU: ~0.05 ms — a handful of indirect calls per pipeline, recorded every frame or once, it doesn't matter.
GPU: the culling compute pass costs ~0.02–0.05 ms for 2000 objects × 4 views. But the front-to-back sort is lost
(unless sorting on the GPU, which is expensive), so early-Z works worse: +0.05–0.2 ms on today's scene pass,
more once PBR and shadows are in. LOD selection has to move to the GPU too. Dev cost: high — culling compute,
compaction, indirect buffers, GPU LOD, and eventually Hi-Z.
And the key observation: in this model the gain comes from the indirect draws, not from the split. Recording
10 indirect calls costs nothing, so cmd_world / cmd_dynamic becomes pointless.
D. No split, recording spread over several threads. One view or one cascade per thread, in secondary command buffers. The Deck has 4 cores / 8 threads. CPU: the same 0.7–2.1 ms of work, but ~0.3–0.7 ms of wall time on 3 threads. GPU: identical to A — sorting, LOD and early-Z are kept. Dev cost: medium — a command pool per thread and explicit tasks. To stay within the engine's rules (predictability first: execution order visible, a crash locatable in under five minutes), the task order must be visible and the assembly of secondary buffers deterministic.
| CPU per frame | GPU added | Dev | |
|---|---|---|---|
| A. No split (current) | 0.7–2.1 ms | 0 | done |
| B. Pre-recorded split | ~0.05 ms | +8–16 ms | medium |
| C. GPU-driven | ~0.05 ms | +0.05–0.2 ms and more | high |
| D. Multithreaded | 0.3–0.7 ms (wall) | 0 | medium |
Decision: D, with C kept in reserve. C isn't abandoned: it moves work from the CPU to the GPU, which is exactly what you want at the end of a project if the CPU is the limit and the GPU has room (say CPU 17 ms / GPU 12 ms → 15 / 14, and 60 fps holds). It gets reopened on a Profile measurement, not before.
The threshold: maxObjectPerCmdBuffer
Multithreading isn't free: each secondary buffer costs a begin / end and re-binding pipeline, sets and viewport
(~5–20 µs on the Deck, not measured). So it only kicks in above a threshold, SAralCreateInfo::maxObjectPerCmdBuffer,
1000 by default.
Why 1000: a draw costs roughly 0.3–1 µs on the Deck and 0.5–2 µs on the Switch 2, whose A78 cores are slower (not measured). 1000 draws on one thread is then 0.5–2 ms on the Switch 2; beyond that, threads take over. The default is set on the Switch 2 (the Deck being the minimum console); a game can change it in its options.
- tasks = passes (view, cascade, cube face); a pass bigger than the threshold is cut into chunks of the threshold;
- below the threshold in total: one thread, zero overhead;
- secondaries executed in the fixed order of the passes (
vkCmdExecuteCommands) → deterministic result; - threads: a persistent pool owned by InGa, created at init, shared by ARAL and other systems (scene graph, culling, streaming) and offered to the game — InGa can refuse a request when the platform's limits are reached;
- one
VkCommandPoolper thread and per frame in flight, reset every frame. Waking a worker costs ~5–20 µs; creating threads every frame would cost ~10–50 µs each, irregularly.
That's step 23. For now one thread is plenty: today's whole aral.flush is 0.35 ms on the Deck.
A side note: static shadow maps are not "recorded once". They are rendered once, with a one-shot command buffer, when they are invalidated.
Shadows: four points to settle
1. Budget and selection
Three different numbers, not one:
- shadows per pixel:
kMaxAralShadowsPerPixel = 8, a visual cap on every platform — past that, one shadow's contrast is about 1/N and invisible; - resident maps, in atlas texels: memory, per platform;
- budget per frame, in passes, per platform. The cost is counted in passes, not lights: a spot = 1, a point light = 6 (reduced to the cube faces whose frustum touches the camera's), a directional = its cascades.
A hard cap kMaxAralShadowPasses = 96 sizes the arrays (96 matrices + atlas rectangles = 7.5 KiB of UBO). The
worst ProjectO scene estimated so far: sun with 3 cascades + 2 rooms × 4 point lights = 51.
When the budget is exceeded, something has to choose:
- local lights get
EShadowCast { eNONE (default), eSCORED, eALWAYS }; directionals abool castShadow; - order: directionals that cast → visible
eALWAYS(sorted by score) → visibleeSCORED(score + ~20 % hysteresis so a light doesn't flicker in and out); - visible = the light's sphere of influence intersects the camera frustum;
- score =
intensity / max(d − r, 1)²; - filling: drop one tile size, otherwise cut and try the next one (a spot can still fit after a refused point light);
eALWAYSonly means "goes first", plus a minimum tile level. Too manyeALWAYSis a level design problem, not something ARAL fixes — but a Debug build warns when one gets cut;- shadow intensity fades in and out when a light enters or leaves the selection;
- casters are culled against the light's volume (sphere, cube face frustum, cascade), never against the camera.
2. Cascades
kMaxAralCascades = 4(C++ and shader: the splits fit in avec4);- per directional light, validated in [1, 4]; the sun defaults to 3, other directionals to 1;
shadowDistance(default 100 m: beyond it, no dynamic shadow);- splits mixing logarithmic and linear,
cascadeSplitLambda= 0.7; - a low-end platform gets 2 cascades through the budget, not through the constant.
3. The sun, driven by the time of day
Since devlog 006, setDirectionalLight(0, …) is refused while the sun follows the clock. So the shadow gets its
own setter: setAstroShadow(bool castShadow, U32 cascades). The clock owns direction and color, the setter owns
the shadow.
- shadow strength ×
smoothstep(0, ~5°, elevation): the sun ↔ moon switch happens with the shadow at 0, no jump; - the shadow direction is quantized to ~0.1° (lighting stays continuous): texel snapping fixes the shimmering when the camera moves, not when the light rotates. A slowly turning sun would make every shadow edge crawl.
4. Toon + soft shadows
The PCF shadow is a soft value s between 0 and 1. In toon mode, multiplying it into the quantized NdotL would
draw a soft gradient in the middle of hard bands, or a thin halo at each band. The rule chosen: the shadow is
quantized separately, with the same condition and the same anti-aliasing as toonNdotL.
float sv = s;
if (bands >= 2.0)
{
float w = max(fwidth(s), softness);
sv = smoothstep(0.5 - w, 0.5 + w, s); // two levels whatever the band count
}
float k = toonNdotL(NdL, bands, softness) * sv; // PBR: k = NdL * s
A uniform branch, no cost in PBR. The shadow also turns off the specular; the sky ambient (SH) is not shadowed.
PCF through sampler2DShadow: 4 reads = 16 texels filtered, 32 reads in the worst case of 8 shadows on a pixel.
Biases (slope + normal offset) are tuned on the toon case: in toon, a bit of acne at s ≈ 0.45 becomes a hard stain.
Static and dynamic, without a merge pass
The old plan had a static map and a dynamic map per light, merged every frame. A merge pass copies the whole map, per light, every frame: 2048² D32 ≈ 32 MB of traffic, ~0.4 ms on the Deck per light (theoretical, from bandwidth). Reading both maps in the shader instead costs two reads per pixel. Both cost about the same as just rendering the full map. So:
One map per light, no merge.
- static / dynamic is about the casters, and a static map is only valid if the light is fixed too (color and intensity don't matter: depth doesn't depend on them);
- fixed light, nothing moving in its volume → static map in cache, 0 per frame;
- a moving caster in the volume → full pass of all casters, that frame only;
- moving light (a carried torch, a swinging lamp) → full pass every frame, no cache, always under budget / selection;
- a static map is re-rendered once (on the pass budget) when the light moves, a static caster in its volume moves or disappears, or a zone loads.
Mobility is the game's call: setInstanceMobility(id, eSTATIC | eDYNAMIC). ARAL guesses nothing. A pillar
that breaks becomes dynamic, the static map is rebuilt without it; the debris go back to static once still, and the
map is rebuilt with their final state. And because a forgotten static object moved by setNodeTRS would leave a
stale shadow silently, a Debug build warns right there.
The sun's zone map
For the sun, a per-zone option: astroStaticStep, in degrees.
- 0 = full CSM (default);
- ∞ = frozen time, a map generated once at load;
- otherwise = regenerated at each step of the sun.
With a zone map, the cascades only render moving casters: the sun goes from ~1–2 ms on the Deck to almost nothing (not measured). A 150 m zone in 4096² is ~3.7 cm per texel, 32 MiB in D16.
The catch is the step size. A step must move the shadow by less than one texel, or the edge jumps:
ΔL = h · Δe / sin²(e) — a 5 m pillar, sun at 45°, 1° → ~17 cm; at 15°, even 0.25° → ~33 cm. Too often at a low
sun. So: two maps per zone (old / new) with a fade over the duration of the step, and the new one regenerated
spread over several frames. A 10-minute day is 0.6°/s; with 2° steps, one regeneration every ~3.3 s.
That one belongs with the zones (later); step 14 does the full CSM.
The rest
- A global switch:
setShadowsEnabled(bool). Some places have no reason for shadows; simpler than walking the lights and remembering their state. Off = no shadow pass, the shader skips the reads (a uniform flag). The atlas stays allocated, invalidations keep being noted, and on re-enable the stale static maps are redone on the pass budget — not all in the same frame. - Several views are independent: each view has its frustum and its cascades (2 views ≈ 2× the shadow work);
local light maps are shared; one pass budget per frame, all views together.
setViewCascades(viewId, n): 3 for the main view, 1 for a minimap or a rear-view mirror, and cascade tiles that follow the viewport size. - The atlas: one D16 image, tile sizes are the game's data (
shadowTileSizes[4], default 256 / 512 / 1024 / 2048, another game could use 512…4096), changed from an options menu withsetShadowQuality(reallocation, deferred destruction). 8192² D16 = 128 MiB: allowed, the game pays. - Alpha-tested casters (foliage, step 15): without a
discardvariant, leaves cast squares. The variant gets its slot in the shadow pipelines from day one, even if empty.
A lesson from the old engine
Before ARAL there was greydog, an older engine of mine, which had PBR and 4-cascade CSM. I fixed it up to have a reference to compare against (a crash at texture load in Release, a Y-flipped bias matrix, missing PCF bias arguments…) and measured it on the same machine. Its shadows were still unstable when the camera moved — that code won't be reused — but it left one clear lesson: pull each cascade's near plane back towards the sun, up to the scene bounds. Otherwise casters outside the camera slice get clipped, and their shadows vanish.
The real comparison comes at the end of step 14: the same greydog scene (ground, red cube, spheres, street lamps, sun + CSM) rebuilt in ARAL.
Where the code is
Step 14.a is done: the whole public side exists, nothing is rendered yet.
- Constants (
limits.h):kMaxAralCascades = 4,kMaxAralShadowPasses = 96,kMaxAralShadowTileLevels = 4,kMaxAralShadowPerPixel = 8, plus bounds for the biases and forshadowDistance(20 m to 1 km). SAralCreateInfogot the shadow fields:shadowsEnabled,shadowAtlasSize(4096),shadowTileSizes[4]+shadowTileLevelCount,astroCastShadow/astroCascades(true, 3),shadowDistance(100 m),cascadeSplitLambda(0.7),shadowDepthBias/shadowSlopeBias/shadowNormalOffset,shadowPcfTaps(4),shadowPassBudget(32).SDirectionalLightgotcascades;SAralViewporttoo (validated insetViewports: a refused viewport list leaves the previous one untouched).- Setters on
IRenderer(no-op on SOLID):setShadowsEnabled,setAstroShadow,setShadowQuality,setShadowBias,setShadowDistance,setShadowPcfTaps,setViewCascades. Each one refuses an invalid value with an error that names it:setShadowQualitychecks powers of two, increasing tile sizes and tiles that fit in the atlas; PCF taps must be 1, 4, 9 or 16; cascades in [1, 4]. - The constructor validates the create info through the same setters, like the toon settings: the default
is set first, then the game's value goes through the setter — a refused value leaves a working default,
never a half-applied one. The pass budget has no setter: fixed at creation, checked against
kMaxAralShadowPasses. - On slot 0,
setAstroShadowandsetDirectionalLight(0, …)write the same fields: the last call wins, written as a comment where it happens.
Next is 14.b, the atlas.
The plan
| 14.a | API first, from ProjectO's main.cpp: create-info fields, constants, setters that refuse bad values — done |
| 14.b | the atlas: D16 image, comparison sampler (border white: outside the map is lit), bound as sampler2DShadow |
| 14.c | one map without cascades: fixed ortho over the scene, depth-only pipeline, casters culled with devlog 004's code |
| 14.d | reading it in pbr.frag: matrices with the tile already folded in, normal offset, PCF, toon rule above |
| 14.e | CSM: log / linear splits, bounding sphere per slice, texel snapping, near pulled back |
| 14.f | the sun driven by the clock: horizon fade, quantized direction |
| 14.g | several views, atlas packing, pass budget, global switch |
| 14.h | milestone: Deck measurement, Blender AgX comparison (finally lit on both sides), greydog comparison |
Each one with its proof, as usual: a RenderDoc capture of the atlas showing the reference scene made in Blender, seen from the sun, no acne in PBR or toon, no shimmering when the camera moves, no jump at the horizon when time runs fast.
Next step
Writing it. The next devlog will have images again.