From Source to Runtime: Asset Cooking in Trident

The resource system covered the runtime end of assets: typed URIs, reference-counted handles, dependency lifetime, and reload. This is the other half of that route. Before ResourceRegistry can load an assets:/ URI, the asset builder has to turn a source file into something the runtime can use.

Asset cooking does more than convert files. Source assets carry editor-friendly data, use source paths, and sometimes produce more than one runtime object. A material graph becomes compiled shader variants, a material definition, and a default material instance. A shader can produce vertex, pixel, compute, and named-variant fragments. A level becomes a binary description of entities and asset references, with none of the editor DOM left in it.

The runtime receives only the cooked outputs.

#Editor vs Runtime Assets

The editor owns source assets through ContentUri values such as engine:/Materials/CopyDepth.mat or game:/Ships/Fighter.meshinst. Their extension says which handler understands the file. The runtime sees the matching extensionless AssetUri, for example assets:/Engine/Materials/CopyDepth.

That extension removal is deliberate. A runtime system asks for a Material; the source format remains in tool code. This also keeps source formats out of game code.

There is a slightly awkward consequence in that Foo.mat and Foo.matgraph would both resolve to assets:/.../Foo. AssetDiscovery rejects that collision before either file can overwrite the other. The build reports both conflicting source files instead of allowing one to silently overwrite the other.

The builder discovers a directory recursively, ignores editor and support files, then selects an IAssetHandler from the source extension. A handler supplies three things:

• the processor which understands the source payload;
• optional sidecar metadata from <asset extension>.metadata;
• the three-byte runtime type header, such as MAT, MSH, T2D, or SHD.

The engine registers its standard handlers directly. Game assemblies can add their own through IAssetBuilderModule, so PaxSolaris can cook its data assets without teaching the engine project what a ship specification is.

#Cooking Transformation

A simple texture processor is a useful baseline. It decodes the source image, applies the requested blend or format conversion, then writes a Texture2DAssetLoader_Data payload through MemoryPack. The runtime texture loader receives that binary structure, not a PNG or an ImageSharp object.

Levels are less simple. The source is a JSON DOM document with folders, editor-specific node types, component property names, and ContentUri references. LevelProcessor walks the document, drops folders, serialises entities and components into Level_Data, and converts asset references to AssetUri values. The runtime gets a compact binary level description and a list of assets it needs to stage. It has no DOM nodes, schemas, or adapters to carry around.

The conversion boundary is where source-only problems are supposed to stop. Import quirks, metadata, data migration, and compiler diagnostics belong on the build side. The runtime loader gets a typed payload and can focus on loading it.

#Multi-fragment outputs

The most interesting processors do not have a one-file-in, one-file-out shape.

A shader processor finds the entry points present in the HLSL and compiles each supported stage. It expands configured flag combinations, honours excluded combinations, and stores reflection data with the compiled bytecode. Each stage and variant is emitted as a named fragment.

A material graph goes a step further. The graph compiler first generates HLSL from the nodes. It then invokes the shader processor for the generated source and the material processor for the generated material. The result includes the compiled shader fragments, the material, and its default material instance under one source asset identity.

Fragments are addressed with URI fragments: assets:/Game/Materials/Example#vertex, #pixel, or #instance. The ordinary output is the default fragment, with no # suffix. This keeps related runtime outputs together without putting source-specific filenames back into the runtime API.

The cooker requires at least one output fragment and rejects duplicate fragment names. This prevents one fragment, such as a pixel shader, silently replacing another with the same name.

#Dependency-aware work queue

Processors can find dependencies only after reading their source. A shader discovers its #include files while it is being compiled. A processor can also ask the cooker to build a ContentUri dependency before it completes.

Cooker holds the work in a channel and runs one worker per logical processor by default. When a processor returns a pending result, the asset is put back in the queue with the dependency task and the dependency URI list. It only runs again after the dependencies complete. A failed dependency fails the waiting asset rather than leaving a stale output that looks valid from the outside.

The work queue has a narrow role, which is to preserve asset dependency order so processors do not read outputs before they exist.

#Incremental Builds

Reprocessing every asset would work, but it would make an editor save feel much heavier than it should. DataCache gives each non-transient source asset a cache record and payload files.

A record captures:

• the processor type, version, and configuration signature;
• a cache signature calculated from the public shape of the processor's runtime data;
• the source file, metadata file, and any additional processor dependencies;
• every output fragment, including its runtime type and payload hash.

The first checks are cheap: timestamps and lengths. If they match, the cache accepts the files without rehashing them. If they do not, it falls back to SHA-256 before declaring the cache stale. That makes ordinary incremental cooks cheap while still catching a content change that kept the same timestamp and size.

There are three outcomes:

1. Data up to date: the cooked output is already present and matches the cache.
2. Cache copy required: the cached payload is valid but the output under Data/ is missing or stale, so the builder restores it without running the processor.
3. Data cook required: an input, processor version, configuration, cache signature, dependency, or cached payload changed, so the processor runs again.

The cache signature deserves a mention. A processor may still have the same version after its runtime data type changed shape. Trident derives a deterministic signature from that type's public fields and properties. Changing a generated data structure invalidates old cache entries without relying on a manual version bump in every processor. Explicit version numbers remain for behavioural changes outside a structural signature.

Transient editor preview assets deliberately skip this cache. Slider drags and graph edits should generate a temporary transient:/ source and output, not leave a permanent pile of cache entries behind.

#Runtime Asset Headers

Every cooked output begins with a fixed 71-byte header:

The header gives ResourceRegistry a fast type check before it hands bytes to a load handler. The metadata also lets tools identify which processor shape produced a file and which exact fragment payload is in it. The cache stores the payload without the header, then recreates the header when it restores an output. That keeps cached data useful while retaining validation at the runtime boundary.

It is intentionally plain. There is no central runtime manifest at present; assets:/ resolves directly into the cooked Data/ tree. The simple file layout has been useful while the engine is still moving around. Packaging and discovery will eventually need more thought.

#Editor reload and previews

The editor runs the asset builder as a long-lived cook server. When a document with a registered runtime type is saved, AssetReloadMonitor asks AssetCookService to cook its ContentUri. On success it schedules a ForceReloadIfAlreadyLoaded call on the game frame provider.

That reload reaches the stable AssetRef sinks from the resource-system post. Existing references keep their identity while the resource registry swaps in the cooked replacement. Editors can therefore use the real cooker and runtime loaders for their previews instead of maintaining a second, almost-the-same preview path. That has saved some duplication, and created a few interesting failures when the real path was not quite as general as I thought.

#What still needs work

The builder is incomplete. Dependency scheduling can be clearer. Error reporting could point back into source documents more consistently. There is no manifest or packaging layer above the cooked Data/ tree yet. Some processors still keep more policy in code than in metadata.

The core separation has held up, though. Tools work with editable source data. Processors turn it into runtime-shaped payloads. The game loads those payloads through typed URIs, with authoring details remaining on the tool side.