Pajama Engine

01 · Vision and product direction

A foundation for worlds that keep evolving

Pajama Engine is the AI-native MMO game engine we are building for persistent, programmable shared worlds. Its Rust runtime combines authoritative simulation, physics, rendering and content delivery. Structured authoring interfaces let developers and agents extend a world through explicit validation and publication. The native client uses winit and wgpu; a planned observation layer connects this foundation to learned models. [1] [2] [3]

The vision is a world that continues to develop through use: players change its terrain, creators introduce new mechanics, and new visual systems give it different forms of expression. Those changes should build on a shared history. An object should remain the same object after a content update; a player's actions should retain their consequences after a server restart.

Space should also extend with exploration: players travel away and return to the structures they built and terrain they excavated. Here, an “infinite world” means a continually extensible, revisitable world operating within finite budgets. The implementation still has a finite terrain domain; terrain status and intent and expansion acceptance identify the remaining work. This extension serves the existing Package and persistent-world goal. [1]

In world-model research, learned representations support prediction and control. Pajama supplies an executable environment and proposes interfaces to such models. The first integration concerns learned visual output conditioned on engine state; learning the world's transition rules is a separate research capability. [R1]

One world, two paths to presentation

Pajama's development combines traditional engine systems with model-generated presentation. The engine maintains entities, collision, gameplay and persistent state. Traditional rendering makes that state immediately playable. The planned model path turns synchronized observations of the same state into generated imagery, with candidates compared by task: Wan for adaptation and cost first, H3 for conditioning and quality, and SANA-Streaming for video transformation. [3]

Consider a wall destroyed during play. Simulation establishes the opening and changes collision; persistence preserves the result. The renderer displays that geometry. A state-conditioned video model would receive observations of the same geometry and camera to generate its appearance. Evaluation checks whether the generated opening actually agrees with the world. This division gives evolving visual systems a stable simulation reference.

From creation to a living world

The core unit of extension is a versioned package: a collection of assets, reusable objects and bounded behavior. Authors prepare a package; the server validates and activates it; clients acquire the matching resources; recovery preserves the associated world and behavior state. This lifecycle gives human and agent authoring a common path from an idea to a persistent capability. [4] [5] [6]

The platform is designed for teams building multiplayer experiences, creators developing reusable content, and researchers working with interactive models. Its long-term value lies in connecting creation, reliable operation and visual generation through the same world representation.

The architecture below explains that foundation. The following status section identifies the implemented systems and current integration work; the development roadmap describes how they grow into a complete runtime and model output layer.

What “AI-native MMO” means here

Agents can act in the world through ordinary authorized player actions, and create for the world through typed, reviewable Package candidates. These two loops meet at one server authority. Models, tools and generators stay outside the simulation tick; their outputs gain effect only through an admitted capability or publication transaction.

MMO names the product direction: shared history, persistent objects, many participants and ongoing content delivery. The current demonstrations establish local multi-client behavior. They do not establish production MMO capacity, seamless world partitioning or a public hosting SLA. The first-party game trenchline is a proof environment for this runtime. World-model presentation remains a separate research track. [1]

02 · System architecture

The architecture connects three responsibilities: creating content, running the world and presenting its state. A package carries a creator's changes into the server. The server applies accepted behavior and physical interactions, then produces the state clients display and checkpoints preserve. The planned model output layer connects to this same presentation boundary.

Three cooperating planes

The simulation plane owns the live world: entity identity, gameplay decisions, physics integration and replication. The control and content plane coordinates admitted releases, authority leases, immutable objects and durable metadata. The presentation plane turns a client's replicated state and verified resources into animation and rendered frames. Rust owns simulation; the TypeScript Edge prototype provides control/content coordination. [7] [8] [9]

Server authority and client presentation are established multiplayer design choices; Unreal Engine documents this division for state, collision and visual effects. Pajama extends that separation to a prospective learned presentation service, while retaining gameplay decisions in the server. This is an architectural precedent, with Pajama's implementation evidence identified separately below. [R2]

This division separates workloads with different lifetimes. A simulation tick needs predictable ordering. A download may be slow or retried. A content update may outlive the request that initiated it. Connecting them through exact identities and bounded handoffs lets each progress under its own scheduling constraints. The boxes describe logical ownership; thread placement is specified separately by each subsystem. [10] [6] [11]

Module definitions and entry points: [12] [13] [14] [15] [16].

Two flows meet at the authoritative world

During ordinary play, accepted input and installed behavior advance world state; physics and other systems produce derived results; replication sends the relevant changes to each client. During content publication, a prepared package candidate becomes eligible for activation at a tick boundary. That changes the definitions available to subsequent gameplay and the catalog clients use to resolve presentation. Both flows participate in the same authoritative lifecycle. [8] [17] [18]

Two agent loops, one commit authority

The fast feedback loop remains native. Remote inference chooses a bounded driving maneuver; the server controller uses current physics feedback. Layout search runs in the cold compiler; the server consumes a verified prefab. A new wheel recipe can use an installed vehicle adapter. A new native physics or renderer capability requires an engine change and compatibility evidence, rather than unrestricted Package code loading.

This separates extension by data and bounded behavior, extension by registered native capability, and proposed external model presentation. The dated implementation documents describe each boundary. Neither an MCP tool nor an authoring agent owns a second world writer.

Architecture priorities

The architecture has a clear strength: world outcomes have one authority, content has an exact identity, and models can connect through an established presentation boundary. The next engineering gains come from cross-system contracts and their validation: take one playable Package through publication and recovery, then export the same world history as trustworthy model input. [1] [19]

These are integration priorities for M1–M6 below. Runtime evidence and each milestone's completion criteria define their implementation status.

03 · Current progress

Edition 1.19 · evidence reviewed September 25, 2026. The runtime now has recorded demonstrations of physical driving, live vehicle additions, bounded constraint construction, real terrain edits, prepared-candidate publication, concurrent author conflicts and checkpoint recovery. The eight new product clips below exercise the same Package boundary with a more consistent native visual treatment.

The pinned architecture sources retain baseline 026acf9 (2026-09-12) and their later individual revisions. September 24–25 additions come from the existing uncommitted engine workspace and retain their own source/artifact hashes. This is a dated integration snapshot, not a claim that one old commit contains all current code. [19] [20] [21]

Current contracts and proposal boundaries are collected in architecture evidence.

Eight practical applications

Each clip is a separate real native run, with an authoritative server, actual local Edge services and two native observation clients. The published view is the main camera. The scenarios use deterministic authored recipes and ordinary client driving inputs, not fresh live Jev calls. The earlier live-model fleet demonstration remains separately attributed below.

Videos run at 1.0× capture time, encoded at 60 fps, for 32–42.5 seconds. Actual main-camera sampling is 59.772–60.005 fps, with maximum recorded intervals of 24–63 ms depending on the run. Missing samples hold the previous image. Recovery contains two captures and an explicitly labeled restart card. These numbers describe the local capture workload, not a production performance SLA.

A running world, captured by the native client

This research station is assembled inside a running multiplayer world. A versioned Package supplies reusable structural parts, a Luau construction policy and a player skill. The native renderer presents the resulting entities, terrain, vegetation and connected players. The case study follows the same run from an empty site to construction and damage.

A package running inside a shared world

A native-client view from the September 21 rehearsal shows the completed building and its surrounding playable world. The next record follows the construction that produced it.

View full image ↗

Provenance

Original 1920 × 1080 presented-swapchain capture, copied unchanged from construction/live-build/left_007.png. The companion public receipt records the camera, capture time and replication tick. The on-screen FPS is a momentary diagnostic, not a benchmark.

Download artifacts 1

• station-complete.png ↓Image · 2.5 MB

  Native client after the package construction completes: a multi-storey building, observation tower, terrain and two connected players.

  SHA-256

  694d8465d5118b380758cbfbceb2509b2db839389108bba816062981f1b1c01f

Evidence manifest ↗

Earlier live-agent and lifecycle demonstrations

Ten independently executed scenarios connect live Jev decisions, Package generation, player actions, construction, collaborative authoring and recovery to observable results. The four-vehicle, dual-vehicle and Jev clips preserve 1.0× real time. The Jev clip separates model round-trip from server receipt latency. The next six clips use 1.25× speed; the final clip is explicitly a still-image lifecycle evidence replay. These separate runs do not represent one uninterrupted server session. Start playback manually. Source downloads require a compatible engine checkout and dependencies; the shared guide includes narration and reproduction commands.

Implementation and integration references: [22] [8] [19] [23] [24] [25] [9] [7].

Integrated results

The September 12 native-client records follow A1 → B2 → A3. A and B are content definitions; 1, 2 and 3 are successive deployment generations. A3 restores A's definition while preserving forward progress in deployment history. Later runs add a stopped HTTPS source, rejoin from a warm disk cache, a deliberately corrupted required asset and recovery into a new server authority epoch. The terrain variant also compares authoritative brush state with typed script state in the same checkpoint. [23] [24]

The useful result is continuity across several lifecycle changes: the clients agree on exact content identity, retained entities remain addressable, and recovery reconstructs the recorded world/content combination. The later construction rehearsal adds player-triggered skill execution and matching cue receipts. Completing the product gate requires combining these results with update and recovery in the supported workflow. At the architecture baseline, retained presentation covers the active and immediate rollback versions; broader historical lineage is a separate workstream. [25] [26] [21]

The next integration focus

The immediate focus is lifetime-managed retained content, a durable observation barrier, stable agent contracts and an author-to-player handoff through the prepared publication workflow. World-model work begins at the observation boundary and develops toward generated presentation. The development roadmap connects these workstreams. [19] [27]

Architecture records for this edition

The prepared-authoring receipts establish a concrete workflow beyond standalone script publication. They use primitive prefabs and two protocol clients. The eight native clips add presentation and application evidence in separate worlds; combining them into one universal lifecycle proof would overstate their scope.

04 · Design principles

Preserve meaning across implementation changes

A world's durable identity consists of its documents, stable entity IDs, exact package references and authoritative state. The engine can rebuild a physics body, replace a decoded mesh or allocate a new GPU buffer while preserving that identity. This separation allows runtime optimizations to evolve independently from saved-world compatibility. [22] [1]

Give every authoritative change a clear owner

One simulation authority orders changes to a world. In leased operation, its epoch identifies the current writer lifetime and fences stale writers. This gives gameplay, publication and persistence a common basis for deciding whether an operation is still allowed to commit. Storage and delivery services can scale independently around that authority. [28] [7]

Treat a release as a lifecycle

A usable content release includes validation, authorization, activation, client delivery and recovery. Each stage answers a different question: are the bytes valid, is this world allowed to use them, which definition is active, can the client present it, and can the server restore it? Keeping these states explicit makes failures diagnosable and avoids collapsing several meanings into a single “upload succeeded” result. [17] [6] [10]

Make resource identity explicit

A package revision and its resource hashes determine what the client needs. Delivery locations can change without changing those identities. The cache verifies bytes; the presentation coordinator publishes a complete compatible resource generation. This supports asynchronous loading while giving late or stale work an unambiguous owner. [29] [9]

Make extensibility observable and bounded

Human and agent authors discover the supported schema, operate within explicit scopes and receive structured results. Sandboxed behavior reads immutable input and returns typed commands. Resource budgets and validation apply at the boundary, where the engine can explain a rejection before it becomes a runtime inconsistency. [30] [14]

05 · World data and object lifetime

A document describes the world; runtime structures execute it

WorldDoc stores serializable world data, including entities and terrain. Runtime World adds indexes, change tracking and lifecycle information needed for execution. EntityId identifies an entity across external contracts. Internal solver, ECS and renderer handles serve the subsystem that allocated them. [31] [22] [32]

The shared contract is semantic: a stored entity, a replicated update and an authoring operation refer to the same entity and component meaning. Their optimal memory layouts differ. Physics needs collision structures; rendering needs batches and resource handles; persistence needs stable, interpretable data. Deriving those views from explicit identities keeps the storage model independent of frame-loop optimizations. [22] [15] [16]

The persistence and package contracts define these distinctions. [1] [6] [9]

Change tracking connects mutation to replication

World::touch advances the world's mutation sequence and marks a full change. A stamp therefore gives an update a position in the world's mutation history; it is more than a per-entity edit count. Dirty-state classification then supports the replication work needed for affected clients. [33] [18]

The simulation layer uses bevy_ecs for storage/scheduling tasks. The server's concrete tick remains the source for cross-system execution order, allowing individual subsystems to move toward data-oriented organization without requiring a wholesale engine migration. [34] [8]

06 · Package identity and publication

What a package identifies

A package binds an ID, an exact authored revision and a digest. Its manifest references assets, prefabs, behavior and exact dependencies. The dependency closure is the complete set of packages required to interpret that release. Resolving the whole set before activation prevents a release from depending on whichever version happens to be found later. [4]

Several version numbers coexist because they answer different questions. The package format determines how bytes are decoded. A package revision identifies authored content. The deployment generation records activation history. Behavior ABI and component schema define execution/data contracts. Catalog protocol defines what the client receives. Frozen document decoders and explicit adapters preserve those meanings as the engine evolves. [4]

From reviewed bytes to a live capability

The current prepared, fenced Package path is: freeze source and expected generation → cold-stage exact bytes → inspect candidate digest → realm admission and semantic preflight → prepare the composite checkpoint and fenced head CAS → reconcile and activate at the authoritative tick → observe the exact Catalog and per-client readiness. The interactive lifecycle below refers to this delivery-aware path. [13] [5] [17] [6] [8] [35]

authoring_prepare_package binds request ID, canonical input, compiler hash and target. authoring_candidate_status rehashes the sealed inventory. authoring_publish_prepared_package verifies the requested candidate digest and submits those bytes without recompilation. The compatibility one-step publisher uses the same implementation. Preparation has no live-world side effect and is not a world-admission dry-run or physics/visual preview. Prepared authoring contract.

The runner has bounded, resumable attempts and an immutable submission intent. A lost reply remains unknown: reconcile or retry the same request and digest using existing release deduplication. Changing a candidate or its expected generation requires a new request. submitted does not mean active; an exact active generation does not prove every client's assets are ready.

Handling a concurrent edit

Suppose two authors both prepare from generation 12. If one release advances the generation, the other's expected base is stale. The intended workflow is to read the new status, reconsider the candidate against that state and submit a newly authorized request. This makes collaboration explicit and prevents a delayed request from silently replacing a more recent decision. The generation check is implemented in the shared publication service. [17]

The current local package_put V1 input uses the composite writer but carries no Delivery Index. Delivery-aware release inputs carry the additional exact resource-location information required for the complete client closure. The roadmap tracks those input capabilities separately from the shared commit mechanism. [36]

07 · Persistence, recovery and rollback

Recover a state that actually existed

A content definition, the script state executing it and the world effects it produced are related. Loading each subsystem's independently latest record can combine incompatible moments. Pajama's composite checkpoint provides one recovery root for the corresponding world, package/admission and recoverable script state. The coordinator owns capture, immutable preparation, head CAS and reconciliation. [6] [37]

The checkpoint head is the committed reference identifying the selected durable snapshot. Preparing or uploading another snapshot alone does not move that reference. This distinction gives restart a clear starting point and allows an interrupted writer to reason about whether its exact candidate became authoritative. [38] [37]

Local and leased operation

In process-local mode, one store owns its root and operating-system lock for the server lifetime. Immutable objects and an epoch-free local head form the recovery namespace. Existing committed state takes precedence over authored seed inputs; corruption is surfaced as a recovery error. [38] [39]

In leased mode, authority is bound to a world/shard/server subject and an epoch. An epoch identifies a writer lifetime. A replacement authority receives a new one, allowing storage and coordination to reject operations from its predecessor. The Edge prototype serializes these lease/head decisions while the Rust server executes the simulation. [28] [7]

Lost acknowledgements are a recoverable state

A storage operation can complete while its response is lost. The coordinator must then compare the observed head with the exact base and candidate it retained. The candidate indicates that the intended write committed; the unchanged base supports the specified retry path; a third, missing or corrupt state requires reconciliation or failure handling. Ownership of the prepared operation survives uncertainty, so callers do not have to guess from a timeout. [38] [37]

Rollback preserves a forward-moving history

This example separates content revision from deployment history. Restoring A can require migrating current typed behavior state into A's contract. World effects that already committed remain part of the world: changing them requires a separately authorized migration or compensating command. That policy preserves player history while allowing the definitions governing future behavior to change. [1] [40]

Current retained presentation is bounded to the active and immediate rollback cuts. M2 addresses wider historical-instance policy, recovery-source retention and durable garbage collection. [26]

Declare how persistent instances evolve

M2 will apply the product contract's pin/rebase choice to each Package-owned instance type. Pinning retains its creation revision and resources. Rebasing adopts a new definition under a validated policy; state changes require declared source/target schemas, typed migration and rollback compatibility. Failed preflight must leave no partial migration effects. [1] [14]

For example, changing a wall's default health requires an explicit decision about an existing damaged wall: retain its old rules or preserve damage through a specified conversion. Downloading an asset or reconstructing a physics object must not make that choice implicitly. Collection must retain revisions referenced by live instances and supported recovery points. Restart and cleanup after A1 → B2 → A3 will test this contract. [26]

Visibility and durability are different observations

Ordinary world changes may become visible before the periodic checkpoint contains them. The new recovery fixture reads the fenced checkpoint head, verifies referenced blob hashes and lengths, and confirms all 15 scene identities and transforms before stopping the authority. Fresh clients then observe the same scene under epoch 2. A screenshot or an active Package receipt alone would not prove this later construction was saved.

A supported durable observation barrier remains a priority: callers should correlate a requested world result with a durable cut without inspecting internal storage. Temporary driving sessions and intents are not checkpointed; recovery preserves vehicle/road content pins but requires new authorization and fresh observations. The demonstration does not claim automatic driver resumption. Recovery evidence.

08 · Content delivery and client readiness

Three identities connect authoring to pixels

The package reference identifies the authored definition. The content digest identifies the required bytes. The presentation generation identifies the compatible set of resources currently exposed by one client. Keeping these identities separate lets the engine move bytes between delivery locations, reuse verified cache entries and rebuild GPU resources without changing the authored content. [4] [29] [9]

The client pipeline is: catalog → Delivery Index and documents → bounded fetch → verified content-addressed cache → CPU decode → hidden GPU staging → complete presentation generation. The Delivery Index supplies exact descriptors for the selected package closure. Only resources belonging to the current selected state may advance that state's readiness. [10] [41] [9]

Each stage has its own completion rule

These are implemented ownership boundaries in the cache, decoder and presentation coordinator. [29] [41] [9] [42]

Byte limits, queue capacity, concurrency and time budgets keep content loading from consuming unbounded work. Late responses remain associated with the selection that requested them. Missing resources produce a bounded diagnostic placeholder, while terminal failures are observable; an imported asset miss cannot fall through to an unrelated legacy shape. [10] [43]

Atomic publication applies to each client's resource set. Different clients can finish loading at different times and still converge on the same content identity. Server-owned physics and gameplay continue to determine the authoritative outcome independently of local mesh readiness. [9]

The recorded warm-cache offline scenario and targeted corruption test exercise this separation. Normal Catalog V2 availability remains gated while the broader lifecycle and historical-retention work is completed. [23] [9]

Retained visibility is independent of successor readiness

The earlier pink-car regression exposed a lifecycle coupling: old GPU resources existed, but exact retained permission was not available to the resolver until successor hydration finished. The fix allows an exact durably admitted retention grant to authorize already verified resident resources. New resource generations still require complete verification and staging. No lookup falls back to a differently versioned mesh or material. Four-client evidence.

The broader lifetime problem remains. A third replacement can push an old live GLB source outside the active/previous catalog union; the engine correctly rejects that publication. An unrelated publication after that replacement can reach the same limit. The new expansion demo keeps core A1 fixed and evolves a separate B1 → B2 Package. This is a valid workload within current bounds, not a solution for arbitrary history.

The required next contract is a budgeted retained-content closure rooted in live instances, behavior state, supported recovery/rollback cuts and in-flight work, with tested reclamation. New bytes, definition activation and instance migration must remain separate decisions. Current architecture findings.

09 · Simulation, physics and terrain

Order is part of the simulation contract

The tick acquires an authority operation and revalidates it after obtaining shared world state. Pending package activation occurs at the boundary. Installed scripts run before subsequent physics/pose processing and final replication. A content transition therefore has a defined place relative to the gameplay that consumes it. [8]

This ordering also handles waiting: a request may have been authorized when it started but become stale while waiting for a lock. Revalidation checks that permission at the point where authoritative work is about to proceed. [8]

Physics has a dedicated interface and timing policy

Physics isolates solver-specific representations. Rapier is the default implementation; Box3D remains a compatibility implementation. Box3D's world is owned by a dedicated thread, while the default Rapier path executes directly. Each implementation satisfies the interface while retaining its own numerical behavior. [15] [44]

Rapier uses 4 ms internal steps, capped at 16 per call. After reaching the cap, excess accumulated time is dropped. The design bounds the amount of recovery work following a stall, trading some elapsed simulation time for continued responsiveness. Evaluating that tradeoff requires observing skipped deadlines, physics behavior and server timing together. [45]

One terrain change has several consumers

A terrain edit changes the authoritative document, invalidates collision data and produces the relevant causal/network updates through the shared terrain entry point. Rendering and collision derive their own representations from that state, so each has a separate readiness lifecycle. A visible surface alone does not establish that its collision representation has completed rebuilding. [46]

The recorded package-terrain scenario verifies brush/state agreement across clients and checkpoint recovery. Geometric collision accuracy and player-triggered skill effects require their own acceptance cases. [24] [25]

Replication classifies changes and applies interest-management budgets. Client movement prediction improves response time and reconciles against authoritative echoes. These mechanisms should be evaluated together: simulation timing determines when truth advances, replication determines when a client learns it, and prediction determines what the client shows while waiting. [18] [47]

Dynamic vehicles and continuous driving

The four-wheel component declares chassis dimensions and mass, wheel positions/radii, suspension stiffness, damping/travel, acceleration, steering and exact body/wheel assets. A materialized recipe and Package pin belong to each instance. New definitions affect subsequent instances; a live vehicle does not silently receive new mass, collision or wheel geometry. Rapier supplies chassis physics, raycast wheels, suspension, forces and collision. Steering and stability aids target playable off-road driving, not an automotive engineering model. Vehicle and driving contracts.

Jev chooses expiring maneuvers from bounded observations; native server control closes the feedback loop. Session identity includes authority epoch, driver, vehicle incarnation and relevant content pins. Stale observations, expired intents, lost control or unready collision require re-observation or braking. In the live fleet video, four Jevs drive and the fifth selects terrain-skill attacks. Package publication is orchestrated by the recording harness; the video does not establish that two Jevs independently authored their new cars.

Terrain changes without freezing unrelated movement

The explosion hitch was partly a server scheduling problem: Accepted and terminal durability cuts had held the whole simulation, so moving vehicles paused together even while clients rendered. For an isolated, terrain-only skill batch, the engine now retains active terrain while preparing a private candidate and its durable composite cut. Movement and replication continue against the active preimage; effects stay private until activation.

For at most four affected collision tiles, preparation overlaps persistence off-thread. Exact source fingerprints bind the derivatives. At the tick boundary, durable commitment and derivative readiness allow installation, terrain publication and receipt/cue release. Mixed entity/damage batches retain their broader fence, and larger terrain batches use the bounded streaming fallback. This is a specific latency fix with local-run evidence, not a guarantee that every edit is stall-free. Implementation and measured continuity.

The new bridge scenario joins terrain excavation, installed bridge colliders and a normally controlled vehicle crossing. Terrain training separately checks matching brushes and fresh collider acknowledgements. A particle burst alone is never evidence of a physical crater.

Why terrain still has boundaries

The engine does not yet implement a seamlessly expanding world. Source inspection at d96cddf confirms that TerrainParams.extent defines the playable half-size on X/Z. The engine default is 250 metres, a 500 × 500 metre square domain. Validation permits half-sizes of 64–4,096 metres; that is a configuration limit, not measured capacity. Individual demo instances may use different settings. [48] [49] [50]

Server movement paths clamp coordinates to ±extent, and physics ground scheduling bounds tile keys using that extent. Chunk loading therefore already exists within a finite world. Increasing extent retains a finite domain and raises resource demand. [51] [52]

Three boundaries need separate diagnosis: the world domain determines where players may go; the resident/visible region determines which chunks are loaded and shown; the persistence boundary determines what survives leaving. A clipped horizon alone cannot identify the cause. Acceptance should record the actual camera, world parameters, render chunks and collision chunks, distinguishing hard limits, missing distant terrain and late streaming. Fog and distant scenery can improve presentation.

October 2026 update: The values above retain their historical source-audit date. The new 40 km terrain horizon expands visual residency; it does not claim to remove gameplay, collision or persistence boundaries.

Contracts for a continually expanding world

The following requirements are planned; an infinite-terrain acceptance record does not yet exist:

These requirements extend the shared terrain mutation and content recovery contracts. Saves retain accepted world facts; native distant views, generated imagery and caches derive from them. [46] [6] [53]

World Labs' Spark 2.0 (2026-04-14) demonstrates view-dependent LOD, progressive streaming and a fixed GPU page pool. Its residency approach is useful design input. The source concerns 3D Gaussian Splatting rendering; it does not implement Pajama's destructible terrain, collision or persistence. [R3]

10 · Rendering and animation architecture

Extract only the state rendering needs

RenderSnapshot presents the camera, instances and other render-relevant state to the renderer. Simulation and networking keep their own ownership. The renderer can consequently change batching, resource layout or GPU execution without becoming responsible for world mutation. [16] [11]

Bevy's read-only extraction from a main world into a render world provides a related architectural precedent. For Pajama, this motivates a dedicated observation extractor alongside ordinary rendering. A model adapter should consume a captured view without holding mutable simulation access. [R4]

Compile configuration before frame execution

The rendering architecture separates a versioned pipeline document, validation of fields and topology, compilation of typed resource dependencies, and execution of an immutable compiled pipeline. For example, a pass that consumes a resource needs a compatible producer and declared ordering; those relationships belong in validation and compilation before the frame executor uses the plan. [11] [54]

Code supplies registered GPU capabilities, while data composes and configures them. That gives authoring a defined vocabulary and lets performance-sensitive execution remain specialized. Package publication continues to use its admitted content boundary; broader renderer configuration and package permissions are distinct surfaces. [11] [1]

Retire resources with their generation

Package assets cross CPU decode and GPU staging before the coordinator exposes a complete generation. Imported resources have renderer-owned identities rather than borrowed legacy names. When presentation changes, retirement follows those identities, preventing a newly selected package from accidentally resolving a resource belonging to an older allocation. [9] [42]

Replicate intent; evaluate poses locally

Animation separates authoritative intent from client-side skeletal evaluation. The server can determine an entity's action or animation intent, while the client samples and blends the pose required for its current frame. This preserves a gameplay decision across clients while allowing each renderer to evaluate at its own frame rate. [55]

Together, the snapshot boundary, compiled pipeline and resource generations define a practical evolution path: add rendering capability behind stable inputs, profile its cost, and preserve the lifetime rules expected by content delivery.

Latest rendering progress and evidence

As of October 1, 2026, native rendering integrates clearer lake water, shoreline foam, distance-adaptive cloud detail, weather, underwater optics and a 40 km terrain horizon into the existing pipeline. Every image below is an original engine capture. Feature validation, visual comparisons and performance measurements retain their separate scopes. Implementation references are pinned to the records at e2134b8; runtime evidence identifies its actual builds and profiles separately, including measurements timestamped October 2 in UTC. [56]

Water, shoreline foam and cloud detail

Water now uses two bounded 128×128 GPU inverse-FFT slope fields. Unresolved small waves contribute variance to reflection roughness, while long waves, rivers and interaction ripples retain their established paths. Lake transport attenuates both incoming and outgoing light and uses Snell refraction and dielectric Fresnel reflection: shallow water reveals its bed and deeper water progressively absorbs light, rather than relying only on a color change. [57] [58]

Shoreline foam follows wave arrival, backwash and decay. Pores and larger gaps break up the bank instead of forming a uniform white outline. Foam covers the lit water surface, remains visible at grazing angles, and leaves sky reflection and sun highlights visible through gaps. It is a bounded realtime approximation, not a shoreline fluid solver. [59]

Clear water and broken shoreline foam

Matched shoreline views before and after the foam refinement.

View full image ↗

View full image ↗

Provenance

Original 1280 × 720 native renderer PNGs. Same lake, profile, camera and time; this fixture omits kitchen props and characters. It is a visual comparison, not a gameplay FPS measurement.

Download artifacts 3

• render-shore-before.png ↓Image · 1.2 MB

  Before: original shoreline foam.

  SHA-256

  880ea3cbd113e0e50bc19e123c2cdae3b063896d875097457f2fb5e25fb914f6
• render-shore-after.png ↓Image · 1.2 MB

  After: porous foam above the water reflection, with larger gaps along the bank.

  SHA-256

  31d437f161bf70d945bc1be50ba8b94d5520949c0d68a7a416e04ad751699d91
• render-progress-proof.json ↓Data · 26.6 KB

  Public measurement receipt, original image/source-record hashes and explicit scope for all six rendering records.

  SHA-256

  e0ea9e1825d8bba0d33c4e7a0e4e89542243f9f5d8b745eb88a77a07a12f6739

Evidence manifest ↗

Cloud erosion is filtered using distance, viewing angle and the pixel footprint reconstructed from the actual projection. Unresolved detail converges toward mean erosion instead of switching erosion off and filling cloud creases back in. Narrow fields of view retain detail their pixels can resolve. Across three fixed views, mean RGB disagreement between 640×360 and reduced 1280×720 captures fell by approximately 49.7%–64.2%. This measures resolution consistency, not realism or overall superiority to Tidewater; local banding and cotton-like structure remain improvement areas. [60]

Cloud detail follows the pixel footprint

Matched narrow-field horizon captures show the change in distant cloud structure.

View full image ↗

View full image ↗

Provenance

Original native 1280 × 720 PNGs, 30-degree vertical field of view. Weather, exposure and profile are fixed. The numerical comparison measures resolution consistency, not overall realism or superiority to another renderer.

Download artifacts 3

• render-cloud-before.png ↓Image · 631.3 KB

  Before: previous cloud detail filtering.

  SHA-256

  08cf9436ec4419f6bf0ce9aa8dc935ccaacb3e5da543afc3fe782421b7895110
• render-cloud-after.png ↓Image · 610.4 KB

  After: projection-aware, mean-preserving erosion filtering.

  SHA-256

  da303878fc7bca83ee061e6b0e5067d78ebd2876653e4cfd569d7044709137f1
• render-progress-proof.json ↓Data · 26.6 KB

  Public measurement receipt, original image/source-record hashes and explicit scope for all six rendering records.

  SHA-256

  e0ea9e1825d8bba0d33c4e7a0e4e89542243f9f5d8b745eb88a77a07a12f6739

Evidence manifest ↗

Shared weather across wind, rain, snow and lightning

Weather supports clear, cloudy, overcast, fog, rain, thunderstorm, snow and blizzard, with automatic evolution and manual transitions from the current state. The server supplies a shared weather clock. Wind drives cloud travel, vegetation, water waves and precipitation. Rain creates ground/water impacts and wet surfaces; snow accumulates and melts, and static roofs shelter nearby precipitation. A fixed spatial lightning source illuminates clouds, fog, surfaces and water, with distance-delayed thunder. Control continues through the existing revision checks and render-publication path. [61]

Precipitation and lightning have explicit budgets, and refinements reuse existing passes. Puddles, shelter and cloud illumination use realtime approximations. Evidence covers environmental presentation and sound logic; it does not add lightning damage, flooding, ice collision or wind forces on authoritative bodies. [61]

Weather across the same lake

Thunderstorm and snow captures illustrate linked sky, water and precipitation presentation.

View full image ↗

View full image ↗

Provenance

Original native 960 × 540 PNGs from the recorded weather refinement. These are different presets, not a matched A/B pair. Still images do not demonstrate audio, gameplay damage or a frame-rate guarantee.

Download artifacts 3

• render-thunderstorm.png ↓Image · 629.3 KB

  Thunderstorm: lightning over wind-driven lake water.

  SHA-256

  ce92ec45ec143f7e974504fd86837022136ae1797175dc583501250ba751eb53
• render-snow.png ↓Image · 597.2 KB

  Snow: drifting precipitation and accumulated surface snow.

  SHA-256

  828e711d97f4476493ccf75598049aee7ebe291d7a8420aed46515570040d830
• render-progress-proof.json ↓Data · 26.6 KB

  Public measurement receipt, original image/source-record hashes and explicit scope for all six rendering records.

  SHA-256

  e0ea9e1825d8bba0d33c4e7a0e4e89542243f9f5d8b745eb88a77a07a12f6739

Evidence manifest ↗

Crossing the water surface

Underwater membership comes from published physical water triangles and local depth, excluding dry camera positions below the bed. Volume transport reuses the water node with four illumination/shadow segments, producing distance-dependent absorption and scattering, approximate caustics, Snell's window and total internal reflection when looking upward. Existing scene motion and temporal response are preserved without giving the renderer simulation authority. [62]

In six-view RTX 4090 / DX12 comparisons at 960×540, the underwater increment in median water-node time was 0.115–0.180 ms. Above-water RGB pixels were identical with the feature enabled and disabled. Separate checks cover 1,440 GPU numerical cases and the forward path. This record does not include full multiple scattering, fluid simulation, swimming or underwater audio, and is not a browser/mobile frame-rate guarantee. [62]

Underwater transport and the water-to-air interface

Views of the bed and Snell’s window from the existing kitchen lake.

View full image ↗

View full image ↗

Provenance

Original native 960 × 540 PNGs from the six-view optical acceptance. The scene isolates water and terrain. Caustics are a realtime approximation; swimming and underwater audio were outside this change.

Download artifacts 3

• render-underwater-bed.png ↓Image · 418.7 KB

  Bed view: attenuation, scattering and approximate caustics.

  SHA-256

  9551d647c1e534a3f3acd541682a777d93645f705ef11b15c42274b56bc0cb10
• render-underwater-snell.png ↓Image · 412.6 KB

  Upward view: refracted sky within Snell’s window and reflection beyond it.

  SHA-256

  f4e149b8decdf4950a7b98e502f7e8a78038ab5b406deea8752d5b572418c4c8
• render-progress-proof.json ↓Data · 26.6 KB

  Public measurement receipt, original image/source-record hashes and explicit scope for all six rendering records.

  SHA-256

  e0ea9e1825d8bba0d33c4e7a0e4e89542243f9f5d8b745eb88a77a07a12f6739

Evidence manifest ↗

A 40 km visual horizon within streaming budgets

Terrain uses fine nearby meshes, progressively coarser distant levels and asynchronous generation. LOD seams, reversed camera depth, retention of the previous layout and complete-layout publication support the horizon. 40 km describes visual terrain residency, not metre-resolution simulation, vegetation residency or a seamless MMO world across that distance. Terrain edits retain their single authoritative entry point, with collision invalidation, spatial script events and network observation following that edit. [63] [46]

In one native renderer fixture, increasing distance from 4 km to 40 km raised the patch count from 984 to 1,202 and CPU mesh bytes by 8.4%. The near 1 m tier retained 528 patches; allocated terrain/grass GPU arenas remained approximately 704 MiB. A separate asynchronous streaming test took about 13.8 seconds for cold start and 6.25 seconds for the layout phase after a 64 m move, including roughly one second of settled sampling per phase. The horizon is visible, but cold and moving-layout generation remain optimization work; these timings do not establish seamless high-speed travel. [63]

Terrain residency at 4 km and 40 km

The same world and elevated camera show the visual footprint of multiresolution terrain.

View full image ↗

View full image ↗

Provenance

Original native 960 × 540 PNGs from an 8 km altitude camera. The distance sweep changes only terrain distance in one executable. This is visual terrain coverage, not a claim about gameplay extent or realtime movement.

Download artifacts 3

• render-horizon-4km.png ↓Image · 318.3 KB

  4 km terrain distance from the elevated camera.

  SHA-256

  6e3a0d5b66c41c4d5eeaa8afb54be4513dea0272b847aab17370a2786a757b69
• render-horizon-40km.png ↓Image · 606.1 KB

  40 km terrain distance with coarse distant tiers.

  SHA-256

  8ee90de99f39a91e26b157de0e0b4726b027b51a557b488aec8f328deff123f5
• render-progress-proof.json ↓Data · 26.6 KB

  Public measurement receipt, original image/source-record hashes and explicit scope for all six rendering records.

  SHA-256

  e0ea9e1825d8bba0d33c4e7a0e4e89542243f9f5d8b745eb88a77a07a12f6739

Evidence manifest ↗

Four clients: verify the profile actually running

A recent four-client kitchen recording ran at approximately 23 FPS per window. Logs showed the older client rejecting the 40 km profile and retaining engine-default revision 0 instead of the requested revision 35. Default grass and grass shadows substantially increased each window's workload. After rebuilding both ends, a controlled test used the same updated binaries, scene and camera to reconstruct that fallback and compare it with the existing authored revision 36. [56]

Conditions: RTX 4090, DX12, 960×540 per window, a 60 FPS cap, fixed camera, frozen visual time, 20-second warmup and 20-second capture. FPS uses all presented intervals over approximately 19.5 seconds, including capture frames; PNG sampling is only 2 Hz. The four authored windows yielded 4,501 audited frames, each window's p90 interval below 17.9 ms. GPU columns take the median across windows, and concurrent timestamp spans include scheduling effects. The recorded build includes concurrent uncommitted work, identically in both controls; this is separate from the earlier dynamic kitchen recording. [56]

The authored profile retains the 40 km horizon, current water/cloud settings and underwater support, without an additional background FPS throttle or output-resolution reduction. Capture readiness now requires a successful application and matching requested/actual revisions, preventing fallback imagery from being certified as ready. This establishes improvement in the fixed scene, not 60 FPS acceptance for arbitrary combat, moving terrain streams or long four-player sessions. Further gates cover movement continuity, low-memory devices, browser performance and complete gameplay workloads. [56]

Four clients with the correct authored profile

Window 0 of each four-client control: reconstructed fallback versus authored revision 36.

View full image ↗

View full image ↗

Provenance

Original 960 × 540 presented-window PNGs. Both controls use identical updated binaries, fixed camera, frozen visual time and a 60 FPS cap. Complete interval counts, build hashes and concurrent uncommitted-build provenance are retained in the public receipt. This is a short stationary experiment, not long-session gameplay acceptance.

Download artifacts 3

• render-four-fallback.png ↓Image · 914.3 KB

  Reconstructed fallback: expensive default grass and shadows.

  SHA-256

  2ec5188857a943249defcfe52fc16c0a4b1980d00568bb5b1179bf25427a2502
• render-four-authored.png ↓Image · 1.1 MB

  Authored revision 36: the 40 km horizon and current water/cloud settings retained.

  SHA-256

  ce29b635406a9d8692afaec8a0fee987599f35b11a351e2e774c5bf07c112be1
• render-progress-proof.json ↓Data · 26.6 KB

  Public measurement receipt, original image/source-record hashes and explicit scope for all six rendering records.

  SHA-256

  e0ea9e1825d8bba0d33c4e7a0e4e89542243f9f5d8b745eb88a77a07a12f6739

Evidence manifest ↗

11 · Human and agent authoring

Discover the connected runtime's contract

The registry describes supported commands and components. MCP exposes scoped tools using that contract, with live discovery identifying what the connected server accepts. The default scopes are read-only core,scripting,content; write scopes are explicit operator choices. The compiled registry remains useful for offline inspection, while live calls require a freshly validated runtime contract. [64] [65] [30]

This matters for long-running projects: an authoring tool and a world server may be upgraded at different times. Discovery gives the tool a way to select supported operations, understand limits and report a capability mismatch before attempting a change.

MCP standardizes tool discovery, input schemas and invocation. World-specific authorization, transaction semantics and resource limits remain responsibilities of the engine and its host. Pajama therefore connects tools to existing admission and publication rules. [R5]

Keep generated behavior inspectable

Luau receives immutable frames and returns typed commands checked by the host. The recoverable ABI2 also makes behavior state explicit: state initialization, migration and execution cross a bounded typed boundary. This allows recovery to associate behavior state with its exact artifact and world effects rather than relying on an opaque VM heap. [14]

Luau's sandboxing guidance supports restricting exposed libraries and isolating scripts, while leaving memory limits, interruption and the safety of host APIs to the embedding application. Pajama's typed commands and execution budgets address that host boundary; the language sandbox alone does not establish gameplay authority. [R6]

An agent can therefore propose content and behavior through the same constrained interfaces used by a human-operated tool. The engine remains responsible for admission, version checks, execution budgets and authoritative effects. This is the practical meaning of AI-native authoring in the project. [1] [14]

Per-invocation budgets bound one script execution; continuous creation also needs to bound accumulated results. M5 will add package/world accounting for live instances, persistent-state bytes and outstanding work, with reservation and validation at typed-command commit. Exhaustion should produce attributable rejections; accounting must cover deletion, migration and recovery. Acceptance tests will run slow, sustained construction to its budget boundary and verify the world-wide limit across multiple packages. [14] [1]

Constraint generation is a compiler pipeline

LayoutSpecV1 → bounded search → independently verified placements → ordinary Prefab → prepared Package. The current language uses an integer grid with hard bounds and non-overlap, plus adjacency, separation, group connectivity and excluded rectangles. Compiler identity, specification hash, seed and exact placements are part of authored provenance. Recovery restores committed results instead of rerunning the solver.

Current bounds are a 64 × 64 grid, 32 nodes/prototypes, 128 rules and at most 1,000,000 attempted placements. BudgetExceeded means incomplete search; Unsatisfiable means the finite search was exhausted. A declared footprint does not prove arbitrary mesh clearance, walkability, structural stability or live terrain support. The camp demo adds footprint-clearance checks for its own asset set. Generation contract and native application.

The next domains should add typed hard constraints, soft objectives, budgets and useful diagnostics separately: terrain support, clearance, access and road grade first. World-dependent inputs need snapshot identity and authoritative precondition checks before placement. Expensive solving stays outside the simulation lock. This is an extensible compilation approach, not a claim that one universal DSL already solves every generative system.

Observe the result as part of the operation

The existing script author runner discovers the live ABI, intersects requested commands with operator ceilings, dry-runs, publishes with revision checks and verifies durable activation through a fresh connection. The current prepared Package runner adds cold preparation, exact-candidate publication and authoritative reconciliation; M5 still owns preview, durable orchestration and broader lifecycle qualification. [66] [67]

A lost network response produces an explicit outcome: not_sent is eligible for safe automatic retry; outcome_unknown requires observation and reconciliation. Visual capture adds another view of the result. A native headless frame can reveal a placeholder or wrong presentation, while authoritative queries verify state and recovery tests verify persistence. A useful transcript combines these observations under the same release identity. [68] [69]

The current prepared runner correlates candidate and release identity; the broader integration contract must also use operation identity and candidate digest to correlate authority lifetime, expected/actual deployment generation, activation tick, durable checkpoint and each client's catalog/resource readiness receipts. Accepted, active, durable and client-ready answer different questions; the trace will retain each result and any missing evidence. After a timeout, query the same operation's outcome; any retry must still pass the existing deduplication and version checks. Lost activation responses, interruption before durability and a delayed client will be explicit failure scenarios. Durable Package activation also does not replace recovery tests for subsequent gameplay effects. [17] [6] [67]

An Agent authors and activates a new behavior

In a fresh September 21 run, an AI agent inspected the live script contract and authored a bounded three-stage rescue-platform assembly. A recording harness used the engine's WebSocket admin API to discover capabilities, submit a dry-run, publish the same candidate with revision checks, and query the result through fresh connections. The retained trace contains actual requests and responses; the generated Luau source is also available.

The observations progress through 0, 1, 2 and 3 assembled sections. The runtime records 3 committed batches, 6 commands, zero faults and zero host rejections. The activated artifact matches the preflight digest; the in-memory and persisted deployment generations both reach 2, with disk_state: exact. This validates the authored behavior and durable activation for this run.

{"schema_version":1,"seq":5,"at":"2026-09-21T13:47:13.855Z","kind":"request","phase":"describe","request":{"cmd":"describe"}}
{"schema_version":1,"seq":6,"at":"2026-09-21T13:47:13.861Z","kind":"response","phase":"describe","response":{"ok":true,"data":{"engine":"pajama-engine","version":"0.1.0"}}}
{"schema_version":1,"seq":7,"at":"2026-09-21T13:47:13.862Z","kind":"request","phase":"script_describe","request":{"cmd":"script_describe"}}
{"schema_version":1,"seq":8,"at":"2026-09-21T13:47:13.865Z","kind":"response","phase":"script_describe","response":{"ok":true,"data":{"runtime_mode":"process_local_abi1","command_contract_v2":{"commands":[{"kind":"set_data_number","capability":"entity_data"},{"kind":"build_prefab","capability":"build"}]}}}}
{"schema_version":1,"seq":9,"at":"2026-09-21T13:47:13.866Z","kind":"request","phase":"script_status","request":{"cmd":"script_status"}}
{"schema_version":1,"seq":10,"at":"2026-09-21T13:47:13.868Z","kind":"response","phase":"script_status","response":{"ok":true,"data":{"runtime":{"active":null},"deployment":{"enabled":true,"disk_state":"exact","generation":0}}}}
{"schema_version":1,"seq":11,"at":"2026-09-21T13:47:13.869Z","kind":"request","phase":"target_discovery","request":{"cmd":"query","tag":"agent-proof-controller","limit":1}}
{"schema_version":1,"seq":12,"at":"2026-09-21T13:47:13.871Z","kind":"response","phase":"target_discovery","response":{"ok":true,"data":{"count":1,"entities":[{"id":1,"name":"rescue-assembly-controller","data":{"assembled_sections":0}}]}}}
{"schema_version":1,"seq":13,"at":"2026-09-21T13:47:13.872Z","kind":"request","phase":"prefab_discovery","request":{"cmd":"prefabs"}}
{"schema_version":1,"seq":14,"at":"2026-09-21T13:47:13.873Z","kind":"response","phase":"prefab_discovery","response":{"ok":true,"data":{"prefabs":[{"doc":"One bounded rescue-platform section.","entities":1,"name":"rescue-span"}]}}}
{"schema_version":1,"seq":15,"at":"2026-09-21T13:47:51.953Z","kind":"authored_candidate","goal":"Author and publish a new bounded Luau rescue-platform mechanic from the live runtime contract; observe three typed build steps and a controller counter.","command_ceiling":["build_prefab","set_data_number"]}
{"schema_version":1,"seq":18,"at":"2026-09-21T13:47:51.958Z","kind":"request","phase":"dry_run","request":{"name":"ai-rescue-platform","revision":1,"tick_interval":20,"capabilities":["build","entity_data"],"cmd":"script_put","expected_revision":0,"expected_active_artifact_digest":"","expected_deployment_generation":0,"dry_run":true}}
{"schema_version":1,"seq":19,"at":"2026-09-21T13:47:51.962Z","kind":"response","phase":"dry_run","response":{"ok":true,"data":{"dry_run":true,"revision":1,"source_hash":"4b0a8d45998eac26","artifact_digest":"13c449bb0780d769898656cd22a78eb4573f2501b256402f121a7bb67aef29c6","bytecode_bytes":521}}}
{"schema_version":1,"seq":20,"at":"2026-09-21T13:47:51.962Z","kind":"request","phase":"publish","request":{"name":"ai-rescue-platform","revision":1,"tick_interval":20,"capabilities":["build","entity_data"],"cmd":"script_put","expected_revision":0,"expected_active_artifact_digest":"","expected_deployment_generation":0,"dry_run":false}}
{"schema_version":1,"seq":21,"at":"2026-09-21T13:47:51.973Z","kind":"response","phase":"publish","response":{"ok":true,"data":{"dry_run":false,"revision":1,"source_hash":"4b0a8d45998eac26","artifact_digest":"13c449bb0780d769898656cd22a78eb4573f2501b256402f121a7bb67aef29c6"}}}
{"schema_version":1,"seq":23,"at":"2026-09-21T13:47:51.975Z","kind":"response","phase":"observe_progress","response":{"ok":true,"data":{"count":1,"entities":[{"id":1,"data":{"assembled_sections":0}}]}}}
{"schema_version":1,"seq":47,"at":"2026-09-21T13:47:53.033Z","kind":"response","phase":"observe_progress","response":{"ok":true,"data":{"count":1,"entities":[{"id":1,"data":{"assembled_sections":1}}]}}}
{"schema_version":1,"seq":65,"at":"2026-09-21T13:47:53.831Z","kind":"response","phase":"observe_progress","response":{"ok":true,"data":{"count":1,"entities":[{"id":1,"data":{"assembled_sections":2}}]}}}
{"schema_version":1,"seq":89,"at":"2026-09-21T13:47:54.892Z","kind":"response","phase":"observe_progress","response":{"ok":true,"data":{"count":1,"entities":[{"id":1,"data":{"assembled_sections":3}}]}}}
{"schema_version":1,"seq":93,"at":"2026-09-21T13:47:54.896Z","kind":"response","phase":"activation","response":{"ok":true,"data":{"runtime":{"active":{"artifact_digest":"13c449bb0780d769898656cd22a78eb4573f2501b256402f121a7bb67aef29c6","bytecode_bytes":521,"capabilities":["build","entity_data"],"name":"ai-rescue-platform","revision":1,"source_hash":"4b0a8d45998eac26","tick_interval":20},"metrics":{"commands":6,"committed_batches":3,"committed_commands":6,"consecutive_faults":0,"faults":0,"host_rejections":0,"last_committed_tick":820,"last_run_us":53,"max_run_us":235,"memory_bytes":649784,"runs":3}},"deployment":{"disk_state":"exact","generation":2,"persisted_generation":2}}}}

This run exercises the standalone ABI1 script-publication surface. Its local progress cursor does not establish restart continuation. The Package case below exercises versioned content and Package-owned behavior; bringing agent creation through that complete lifecycle is the purpose of M5. [66] [67]

12 · System 1 and System 2: agent roles, coordination and integration

Pajama assigns agents two kinds of work: System 1 selects the next action from existing capabilities; System 2 plans tasks and authors new capabilities. They cooperate through versioned Packages, bounded player intents and verifiable receipts. System 1/System 2 describe engineering roles here, not claims about a model's internal cognition or a fixed speed guarantee. The current System 1 adapter uses Jev; System 2 integration targets Codex and other planners that can call tools. [70] [R7]

What each agent can do

The Jev adapter currently supports up to eight configured skills plus wait and plan. It cannot generate Luau, invent skills or understand the complete rendered scene. Broader NPC tactics, skill ranking and target selection require separate input contracts and acceptance. System 2 is likewise limited by connected tools and authorization: generating code does not establish that it is active in the world. [71] [72] [73]

A stable contract for future agents

The prepared tools are usable building blocks; the following shared third-party agent contract is still proposed. It should preserve agent/world/session identity, granted capabilities, permitted observation scope, observation time/digest, relevant exact Package references, action deadline, cancellation state and budget ownership across reconnects. Action outcomes must distinguish accepted, committed/failed, unknown and observed postcondition.

An unrelated Package addition should not cancel a still-valid pinned driving action. A revoked capability, changed relevant definition or stale observation must cause re-observation or rejection. Inference, asset generation and compilation need bounded durable jobs; they must not hold the world tick. Cancellation after a world commit cannot undo that commit.

The next integrated proof is one author agent creating a capability, publishing through the existing transaction, and a separately authorized acting agent discovering and using it. Include a conflicting candidate, lost response and coordinator restart. The live driving video and deterministic eight-scenario suite provide components of this proof, not completed autonomous creator-to-player orchestration. Proposal and prepared implementation.

Coordination through explicit handoffs

User goal
  +-- Existing skill suffices -> System 1 -> type/confidence/state checks
  |                                        -> player intent -> authority
  +-- New capability / multiple steps -> System 2 -> candidate Package
       -> validation, admission, CAS, checkpoint, tick activation
                                  |
                    Fresh Catalog + approved skill policy
                                  |
                    System 1 selects an available skill

Authoritative result -> replication, cues and receipts -> observe or plan

This is the target coordination flow. Both entry points exist, but needs_planning remains a structured result for the caller. Automatic Codex invocation, a cross-process handoff queue and resumption of interrupted tasks are not yet connected. [70]

For “build a destructible castle, then open an entrance,” the roles divide into five steps:

1. Discover and plan. System 2 reads world_script_describe and content contracts, establishing component counts, resource budgets, destruction behavior and completion criteria. An already approved skill can be used directly when no new capability is needed.
2. Author and validate. System 2 produces Luau and Package candidates, coordinates existing asset generators and verifies binding results. Luau reads immutable frames and returns typed commands; network calls and generation run outside the sandbox.
3. Publish and confirm. Candidates use the existing cold stage, admission, generation CAS and composite-checkpoint path, with tick-boundary activation. A completed job or submitted release does not establish an updated world: query authoritative active for the exact Package identity.
4. Hand over available capabilities. The caller refreshes the Catalog and updates the skill policy according to authorization. System 1 selects an approved entrance-breaking skill only against fresh observations; late decisions for the previous Package revision are discarded.
5. Execute and inspect. The server validates player intent, executes Package behavior, commits collision/terrain/entity changes and replicates them. System 1 consumes the matching execution receipt; System 2 revises plans in response to failure reasons or new user goals.

This example describes coordination design, not an already completed “Codex builds a castle, then Jev breaches it” joint acceptance. Building, character-authoring and Jev proofs come from separate run records. [73] [20] [74]

A proposed handoff record should include the original goal/intent, task ID, world and player-session identity, observation timestamp/digest, exact Package references, approved action scope, remaining time/call budgets, prior request ID and authoritative result. This is a new caller-orchestration contract, not an existing server message schema. Model output and web/chat content cannot expand action permissions.

System 1 currently rechecks session, server identity, character, Catalog, active Package closure, terrain and the public observation digest. It rejects expired or superseded decisions. Confidence guides routing; the engine retains authority, and high confidence does not guarantee semantic correctness. [71] [72] [R8]

Connect System 1: OpenRouter / Jev

Available interface. Run from the engine repository with Node 22+, a Catalog V2 game server and an already published, active character/skill Package. Copy scripts/system-one/config.example.json into an ignored run directory, set the actual game_url, and configure character and skills. The example uses ground-pulse from studio.generated.meshy-hero; a policy entry neither creates nor publishes that Package. [70]

# Supply OPENROUTER_API_KEY through the process environment, not JSON or model context.
node scripts/system-one/cli.mjs target/my-jev-demo/system-one.json target/my-jev-demo/intent.txt

An intent file can request “Dig a small hole beneath my feet using ground pulse once.” The CLI joins as its own player, waits for character selection to replicate, makes at most one model call and prints JSON. Remote game connections require WSS; loopback also permits WS. Reusable Node interfaces are JevProvider, GameClient and SkillDirector.decide(intent). [70]

The adapter uses OpenRouter's native POST /api/alpha/decisions, requests typesafe/jev-1.13, restricts routing to TypeSafe without fallback, and requires the accepted response version typesafe/jev-1.13-20260917. Chat Completions is not a substitute for this contract. Defaults are a 1.5-second timeout, a 2-second observation age, selected probability of at least 0.85 and confidence of at least 0.65. Thresholds still require task-specific calibration. [75] [R9]

Connect System 2: Codex / MCP

Available interface, with a client-publication acceptance gap. Codex first discovers capabilities through read-only engine MCP, then explicitly connects authoring MCP to submit candidates. The following config.toml example requires actual absolute paths in place of G:/pajama-studio/engine, plus a running server and authoring host. [73] [R10]

[mcp_servers.pajama-engine]
command = "cargo"
args = ["run", "-q", "-p", "pajama-mcp", "--", "--require-live", "--scope", "core,scripting,content"]
cwd = "G:/pajama-studio/engine"

[mcp_servers.pajama-authoring]
command = "node"
args = ["G:/pajama-studio/engine/scripts/authoring/mcp.mjs"]
startup_timeout_sec = 30
tool_timeout_sec = 120

[mcp_servers.pajama-authoring.env]
PAJAMA_AUTHORING_CONFIG = "G:/pajama-studio/engine/target/my-authoring-demo/config.json"

Build pajama-mcp first with cargo build -p pajama-mcp in the same environment so initial compilation does not consume the handshake window. Record codex --version and inspect connections with /mcp. For a non-default admin port, add --admin and the actual loopback URL to the read-only MCP arguments. [76]

The current local authoring surface has seven tools: authoring_generate_character, authoring_character_status, authoring_prepare_package, authoring_candidate_status, authoring_publish_prepared_package, authoring_publish_package and authoring_release_status. The three prepared-candidate tools extend the historically cited four-tool interface; their current contract is in the prepared authoring record. A planner prepares files, inspects the sealed digest, submits that exact candidate and polls activation, then inspects the world through authorized queries. Generated character artifacts enter an ordinary Package after binding validation. Behavior-bearing Packages must follow composite-checkpoint publication; separate content/script writes cannot emulate atomic publication. [73]

In the September 21 record, Codex CLI 0.155.1 completed MCP handshake and discovery; automatic approval review rejected its actual publication call. The later prepared-authoring test exercises a separate stdio MCP process and real publication; it does not retroactively establish that client's approval-path acceptance. Engine-state queries, native rendered frames and model visual input are distinct capabilities; this MCP configuration does not automatically stream live video into Codex. [73]

Custom Cloudflare agents: orchestration outside engine authority

Available remote job bridge. services/pajama-author-agent uses the Agents SDK to manage durable jobs. A planner uses author credentials to submit POST /jobs and reads GET /jobs/<id>. Local scripts/authoring/remote-bridge.mjs claims outbound work and calls the same AuthoringRuntime, keeping the admin listener local. Author and bridge credentials are separate; jobs use idempotent IDs and expiring claims. [77]

{
  "id": "observe-agent-demo",
  "name": "world_observe",
  "args": {}
}

This is an existing observation-job body, without credentials. Publication jobs use authoring_publish_package with reviewed Package input in args; a completed job still requires an authoring_release_status check for authoritative activation. The Cloudflare job service does not run an LLM automatically or host Jev's player WebSocket decision loop. Scheduling both agent roles requires further handoff, player-identity authorization and recovery work in the orchestration layer. [77] [73]

Cost, evidence and joint acceptance

Reduce cost first by routing work: explicit waiting and deterministic decisions need no inference; System 1 handles simple existing skills; new capabilities, ambiguity and complex goals go to System 2. The implementation bounds observations, skill count, frequency and calls per director lifetime. Cross-process budgets, durable task queues and a general rule router still require orchestration work. Package publication and physics ticks do not wait for model inference. [71]

In the final September 21 Jev run, three real requests—wait, a Chinese complex goal returned for planning, and a Chinese skill command—took about 180–415 ms each. Returned charges totaled $0.000070938. The real server executed the skill; two player connections received a matching dig brush with a 2 m radius, and the observer received matching cast/impact cues. The first English-only skill description yielded probability 0.78 and no activation. Adding a bilingual description yielded 0.99 without lowering the execution threshold. [74]

This proves a bounded skill loop, not sustained multi-agent autonomy. A later local Wrangler disconnect caused the server to exit after lease renewal failed; that environment failure is retained. Joint acceptance still needs actual Codex publication, automatic needs_planning handoff and budget control, reconciliation after orchestrator restarts, invalidation of old decisions during publication, matching frames on two native clients, and sustained operation through recovery. [70]

13 · AI-assisted engine evolution

Proposed architecture. Pajama should support a durable loop from observed failure to validated change, then retain evidence that helps the next improvement. The external evolution control plane coordinates experiments and releases; the authoritative server continues to own world state. This extends the existing product boundary and does not require a new optimizer inside the tick. [78]

Duan et al.'s September 15 revision distinguishes improving an artifact from improving the process that creates its successors. Reusing a changed improver establishes structural recursion; independently better successors at comparable budgets are a further test. This survey informs our design, not a claim that unrestricted self-improvement is solved. [R11]

Five targets, explicit release boundaries

A common ledger correlates releases without turning them into one activation mechanism. Packages do not load native Rust code. An old binary cannot read a new persistence format without an explicit compatibility adapter. A content rollback restores definitions and validated typed state; it does not erase destruction already committed to the world. The current V1/ABI2 publication split still needs convergence. [78] [4]

Close the loop outside the simulation

World / native clients / CI / authoring receipts
              |
       Reproducible case + exact versions
              |
       System 2: hypothesis and bounded plan
              |
       Isolated candidate -> tests / replay / performance
              |
       Independent verdict -> rejected or inconclusive evidence
              |
       Existing release path -> staging -> scoped rollout
              |
       Authoritative results + retained recipe -> next case

System 1 executes approved skills and supplies bounded outcomes; it can also exercise a staging scene through ordinary player intents. System 2 diagnoses the failure, chooses the responsible layer and authors a candidate. Executable checks and an independently controlled evaluator decide whether to promote it. An author's model-generated verdict cannot substitute for world receipts, regression tests or native frames. [78] [70]

The first implementation can be a repository runner with immutable artifacts and an append-only experiment journal. Cloudflare may later coordinate durable claims and budgets; native builds and GPU trials run on suitable workers. The current Cloudflare authoring bridge is not an engine-build or training executor. No new always-on microservice is required to start.

How each role makes engineering decisions

Use a versioned decision protocol. At each round, provide the agent with the current goal, authorized scope, owner lane, exact source/runtime identities, relevant contracts, a bounded set of prior decisions and actual evidence. Distinguish hard invariants from conditional practices and untested hypotheses. Repository agents can now follow docs/ENGINE_DECISIONS.md through the AGENTS.md routing table and use its decision template. Automated enforcement remains proposed. [79] [80]

These are responsibilities, not a requirement to run six models. One System 2 agent can perform several stages sequentially, with protected evaluation and release rights kept separate. System 1 remains a bounded skill selector. A future fast path may reuse a proven engineering recipe when its preconditions match and required checks still run; the current Jev adapter does not choose architecture or edit engine code. [79]

For a material choice, the proposed protocol applies this order:

1. Identify the product outcome and owner. Read the relevant lane rather than loading the entire repository. Mark disagreement between code, documentation and tests as a gap to investigate.
2. Separate facts from uncertainty. If the cause is unknown, select the cheapest bounded observation that could change the decision. Do not invent confidence percentages or precise gains.
3. Compare plausible options. Include the current baseline, a local change behind the existing interface and a broader alternative when meaningful. Gathering evidence is a valid choice; trivial fixes need no artificial alternatives.
4. Filter before ranking. Reject violations of scope, authority, compatibility, hard budgets or recovery. Compare feasible options by relevance to the product blocker, evidence, benefit, complete cost, affected surface and reversibility.
5. Choose the next action. Return measure, experiment, implement, promote, defer or escalate, with concise rationale and invalidation conditions. Promotion needs independent acceptance of the exact candidate.
6. Stop or re-plan from outcomes. Stop when the gate is met, the budget is spent or another experiment has little decision value. Re-plan after a changed baseline or a counterexample. Routine authorized choices proceed without repeated permission; escalation is for genuinely missing scope, authority or product tradeoffs.

If options trade different benefits and neither dominates, use a small discriminating experiment or obtain the missing product preference. This protocol improves accountability; it does not guarantee a globally optimal choice. Keep a reviewable explanation and evidence, not private reasoning transcripts. [79]

Best practices need applicability conditions

Epic's profiling guidance ties performance budgets to target hardware. Godot's September 15 rendering report shows why an intuitive optimization, including caching, may lose through memory traffic. Google's canary guidance couples limited rollout with controls and representative evaluation. These sources inform methods; the following applications are Pajama proposals. [R12] [R13] [R14]

Rules should carry an ID/version, source, owner, applicability condition, required evidence, validator and invalidation triggers. “Always cache,” “always add threads,” and “always shard” are not valid rules. Native performance continues to use the existing A/B/A and fidelity gates; changing architecture does not waive them. [79] [81]

Example: deciding what a terrain boundary requires

The same symptom supports different choices depending on the actual goal. This is a proposed pre-experiment comparison, not a new measurement:

For the continuous-world goal, first choose measure: bind the scene/source identity and observe boundary behavior, collider/terrain residency and frame/tick/memory cost. Then select the smallest streaming experiment that can test traversal, edits and recovery. If the user's goal is only presentation, the first option can be reasonable. This makes the choice responsive to the goal rather than to a fashionable technique. [79]

Each consequential decision should retain alternatives, rejected reasons, constraints, evidence, total budget, chosen action, stopping conditions and later outcome in a versioned record. Small fixes can keep that information in their change description. A proposed dispatcher verifies versions, budgets and permissions; protected runners bind evidence to the actual artifact and evaluator. Agent-authored passed: true is not a valid attestation. The new guide/template supports this process today, but the dispatcher and attestation service are not implemented. [79]

The AI may propose revisions to best-practice rules as a separate experiment. An authorized policy owner adopts independently validated changes for later rounds; the candidate cannot weaken its own current test. Evaluate the decision policy itself against a frozen baseline on held-out engine tasks, including invalid promotion, unnecessary escalation, inaction, regressions and total cost—not merely whether it filled in every field. [79]

Persistent evidence, bounded execution and recovery

Two stores have different responsibilities: world checkpoints preserve gameplay truth; the evolution ledger preserves experiments and decisions. Link them through exact identities. The proposed coordinator records are:

These are proposed orchestration records, not existing MCP tools or server messages. Persist budget reservations before dispatch. Claims and journal revisions fence stale workers. On restart, reconcile an unknown publication or skill request by its original identity before another effectful attempt; do not restore spent budgets. A changed baseline requires reevaluation. [78]

An experiment copies an authorized checkpoint into a separate world identity, storage namespace and writer lease. Retain seeds, intent/event sequences, tick policy, exact content, model responses and hardware/runtime identity. Recorded responses help reproduce a failure; separate real-provider trials measure model drift and latency. Physics and GPU results are not assumed bitwise identical across machines: compare exact document/event identities where required and declared semantic/visual tolerances elsewhere. Keep credentials and raw private conversations out of evidence exports.

Measure improvements without rewarding regressions

Freeze acceptance criteria for each round. The candidate author can propose new tests but cannot remove protected cases, lower thresholds or rewrite held-out scoring during the same evaluation. Correctness, authority, compatibility and recovery are hard gates. Compare completion, regressions, tail latency, memory and cost separately; a higher aggregate score cannot compensate for corrupted state. [78]

For native performance, reuse the existing tools/perf/baseline.py and tools/perf/compare.py workflow. Its A/B/A controls bind source, binaries, workload and presentation state, with visual-fidelity checks. Default acceptance requires at least 3% median improvement, at most 2% p95 regression and less than 8% control drift. Inconclusive results do not justify a release. General CI checks engine correctness; acceptance-grade timing requires a stable runner. Missing private fixture coverage remains unverified. [81] [82]

Reduce experiment cost with reproduction deduplication, cached immutable builds, small deterministic tests before expensive native/provider trials, event-triggered diagnosis and bounded candidate search. Report total cost per validated improvement, including failed attempts, evaluation, model calls, compute and human review. For changes to the improver itself, compare old and new versions from the same engine baseline on unseen tasks with the same model access and total budget. Confirm that retained knowledge changes a later decision; merely storing notes is insufficient. [78]

First proofs: agent policy and continuous-world engineering

The Jev bilingual-description fix is a useful seed: its small recorded sample improved selection without lowering the execution threshold. It was human-directed, not an autonomous evolution demonstration. Build an independently labelled test family covering Chinese/English paraphrases, negation, waiting, complex plans, stale Catalogs, cancellations and uncertain activations. Compare correct actions and correct abstentions, then verify accepted choices through two player connections. Retain the winning policy and prove its reuse on a new case. [74] [78]

For an engine-level case, travel toward the terrain boundary and distinguish an intentional world limit from missing collision, missing chunks or a visual cliff. A continuous-world candidate must verify traversal, edits, revisit, two-client agreement and reconnect/restart while bounding memory and tail latency. Hiding the horizon alone cannot pass. Coordinate/extent changes require explicit compatibility work; the agent must not simply remove the movement clamp. This turns the world-expansion direction into a sequence of testable changes. [78] [48] [51]

The same exact-state scene can become a world-model experiment, with splits by world/scene/trajectory to avoid adjacent-frame leakage. Improve capture selection, caching or serving first where measurements justify it; training new weights remains an offline candidate with its own cost and causal-consistency checks. H3/Wan selection continues to follow the shared-scene comparison.

Increase autonomy only after the corresponding proof

The engine has useful authority, Package, CI, performance and bounded-agent foundations. The unified evolution coordinator, protected evaluation service, automatic curriculum and autonomous binary rollout are not implemented. This chapter proposes their contracts; it adds no runtime acceptance claim. Start with a named product-gate failure or measured limit, keeping the coherent Package lifecycle as the shipping priority. [78]

14 · Dynamic gameplay through a versioned Package

The research-station rehearsal makes the Package boundary tangible: one release carries a structural kit, a construction policy, a damage rule and presentation cues into a running world. The recorded release is studio.generated.meshy-hero, revision 2, digest beginning 93d02749. Two native observers are connected before the new construction begins. [83] [20]

Author reusable parts and executable rules

The Package defines 17 structural part types, including floors, columns, walls, stairs and tower sections. Each part carries a primitive visual shape, static collision and a breakable-block component. The build-fortress Luau policy computes placement from a small set of dimensions, then emits bounded batches of typed build_prefab commands. Its recoverable state records construction progress. The runtime resolves every part through the admitted Package revision. [83] [14]

This is the practical scope of dynamic gameplay authoring: an author supplies data and bounded policy, and the running engine turns them into interactive world state. A generation system can propose those same documents. The station is a recorded execution of an authored Package; the separate Agent trace demonstrates live AI authorship and script publication.

Publish into an already running world

The rehearsal starts from the character-only release and observes an empty construction site. The authoring service submits the new Package against the expected generation. A subsequent authoritative status query records generation 2, catalog revision 2, and the exact revision-2 digest. The construction history records 0 → 399 Package-owned blocks, including intermediate counts. This links publication identity to an observable world change. [20] [17]

From package behavior to constructed, damageable objects

The same native view records the open site, foundation and growing structure. The recorded package creates 399 building parts; a separate breach-wall interaction in the same run reduces a selected wall from 180 HP to removal, observed by both protocol peers.

View full image ↗

View full image ↗

View full image ↗

Provenance

Recorded September 21, 2026 in target/complex-building-evidence/construction. Images are unchanged originals; JSON is an explicit public field projection and the Luau files are exact source strings from build-live.json. This deterministic rehearsal is distinct from an LLM tool-call transcript. Implementation reference fe4d401 is available; the artifacts do not embed the executable’s exact commit.

Read evidence excerpt

Formatted summary derived from package-rehearsal.json / 根据公开回执整理的摘要
2026-09-21 · deterministic package rehearsal / 确定性内容包演练
package: studio.generated.meshy-hero @ revision 2
publication: request build-live → generation 2 / catalog 2
construction history: 0 → 399 parts
typed behavior: build_prefab batches; persisted cursor
breach-wall requests: 91001 / 91002 / 91003
target 994 HP: 180 → 120 → 60 → removed
building parts: 399 → 398
requesting and observing protocol peers: cue events and remove 994 recorded
Not an LLM conversation or raw tool-call log / 非 LLM 对话或原始工具调用日志

Download artifacts 6

• station-before.png ↓Image · 2.6 MB

  Before publication: the same native client shows the open site; 27 entities are present in its captured view.

  SHA-256

  504884b9db59939fb73929a024cbce1e0f010f460e31967b1435c76dd6d2f6ce
• station-foundation.png ↓Image · 2.6 MB

  Construction begins: the package behavior places the foundation while the clients remain connected.

  SHA-256

  fd94695eed81e9501c59df2f3ec0a1f687d7904b4eaafd5f1ae0c994a71a2115
• station-in-progress.png ↓Image · 2.6 MB

  Construction in progress: floors, columns and walls appear through bounded batches of typed build commands.

  SHA-256

  2cbe867c7653cac3e92a018d69093534b421cc7a1717d1cddd0ed8a98f012610
• package-rehearsal.json ↓Data · 21.7 KB

  Public field projection of the recorded publication, construction, damage, cue and removal receipts, with capture metadata and source-file hashes.

  SHA-256

  510f28691846a31a3de06cc717fa6ed07796ec0289fcff8236710fd2b18efee1
• build-fortress.luau.txt ↓Trace · 4.4 KB

  Exact build-fortress Luau source from this run’s submitted package bundle: procedural layout, typed build_prefab commands and a persisted construction cursor.

  SHA-256

  892c8f84d4817844a966f04474e8207713e6c0a964923e8bfcf42222a6280e8c
• breach-wall.luau.txt ↓Trace · 1.2 KB

  Exact breach-wall Luau source from the same bundle: validates the authoritative target and returns a typed 60-point damage command.

  SHA-256

  15d2fb24760ce376a657fcaea7e6efe1daf627c7e3d3a87f611f0dbfa6c8b6cf

Evidence manifest ↗

Turn generated structure into gameplay

The same Package binds the breach-wall skill to bounded behavior. A player sends a skill intent; the server's targeting path selects the hit, and the behavior checks the immutable event/frame before proposing 60 damage to the eligible block. The block's initial health is 180. The accompanying probe sends player-protocol requests without world-edit authority. [83] [21]

The downloadable receipt excerpt preserves the Package identity, sampled construction history, three cast outcomes and both clients' relevant messages. Cue receipts establish event delivery; the wide construction screenshots establish visible scene assembly. Pixel-level effect attribution and the full update/recovery release gate remain separate checks.

Carry the capability through its lifecycle

A generated mechanic becomes useful beyond a demonstration when its content, behavior state and lasting effects survive publication changes, reconnect and recovery. The September 12 terrain/checkpoint run and this later construction/skill run provide different parts of that evidence. M1–M3 bring these capabilities into a repeatable supported workflow, including retained instances and exact-version presentation. [24] [23] [26] [40]

15 · Verification and performance methodology

Verify properties at the level where they matter

The release objective is one repeatable black-box scenario covering a real render asset, prefab, authoritative mechanic and matching presentation across update, reconnect, restart and rollback. Component tests establish local rules; integration and native-client tests establish that those rules cooperate under the actual lifecycle. [19]

This matrix organizes the product completion criteria and the recorded native scenarios into a review method. [1] [23] [25] [53]

The native runner is services/pajama-edge/test/native_catalog_v2_two_client_e2e.mjs. Its recorded scenarios cover two independent native clients, update/rollback, offline rejoin, corruption and server recovery. Receipt counts, screenshots and manual inspection serve different purposes; the roadmap separately tracks stronger per-entity pixel attribution. [84] [23]

Build a reproducible performance baseline

Performance runs should pin the engine revision, content hashes, workload, configuration and machine/OS/GPU. Separate a first join with an empty cache from a repeat join with a warm cache. Measure steady play, publication during play and recovery as different phases, so the cost of loading or durability is visible rather than hidden in an overall average.

These measurement priorities follow the product goal. Capacity targets and budgets should be justified by recorded workloads; the current whitepaper does not publish a player-count or latency SLA. [1]

Document provenance

Core architecture descriptions reference engine 026acf9 (2026-09-12). The construction/skill rehearsal references later authoring implementation fe4d401; the agent integration chapter is pinned to 382ca36; world-model architecture uses a separate planning revision. Native captures, authoring traces and Jev protocol acceptance retain their individual provenance and file digests. The references section collects source coordinates and revisions; the evidence manifest identifies downloadable artifacts. Visual world-model integration remains planned work.

This edition adds source inspection of terrain extent, validation, movement limits and ground scheduling at d96cddf. This bounded review neither advances the entire core baseline nor supplies new infinite-world runtime evidence. External references were checked through 2026-09-21; their publication dates and scope are retained below.

What the latest videos measure

The eight main-camera recordings meet the suite's predeclared floor of 59 actual fps and ceiling of 100 ms per sampled interval. They retain frame manifests and separate measured capture cadence from the 60 fps MP4 encoding rate. Main views are 1600 × 900, except the 1440 × 810 recovery footage; a second native observer is captured at 15 Hz. Full MP4 decoding and byte identity checks passed during delivery.

The bridge video has seven unrepresented sample slots and the convoy has three; the final accepted clips are not all zero-drop recordings. Recovery has two separately validated 1,200-frame captures. Failed and superseded runs remain diagnostic evidence. The optional lossless capture staging format and render quiescence reduced recording overhead; this does not measure general multiplayer scalability.

Original beveled building/utility meshes, shared materials and daylight, reduced ground noise, deliberate cameras and story captions improve the native presentation. Vehicles retain Kenney CC0 attribution. Buildings have exterior solid colliders; this suite does not claim walkable interiors or ecological tree simulation. Video hashes and dated records.

This edition reuses recorded acceptance. It validates the website and published artifacts separately; it does not rerun paid inference or relabel an edited video as a new engine benchmark. Historical citations retain their original commits, and current source inspection is explicitly hash-bound. External research remains dated to its previous review rather than being silently presented as newly verified.

16 · World-model integration

Explicit state, generated presentation

The proposed integration has a precise responsibility boundary: the engine computes authoritative world transitions; the output model synthesizes a visual observation from permitted conditions. Here, world-model-friendly describes an interface and evaluation strategy. The planned bridge aligns actions, physical outcomes, camera views and generated frames with an explicit state history. [3]

The intended dataflow can be written as three operations:

S(t+1) = EngineStep(S(t), accepted_actions(t))
C(t)   = Observe(S(t), camera(t), view_permissions, content_revision)
I(t)   = Model(conditions_available_by(t), appearance, session_history)

S is authoritative state; C is a declared observation projection; I is generated imagery. The engine step also depends on pinned code, configuration and numerical behavior. Capturing its results does not imply bit-identical solver replay across platforms. The model may approximate appearance or motion; its pixels do not commit world changes. Generated gameplay proposals still pass through typed commands or package admission. [15] [4] [85]

The native path uses RenderSnapshot and wgpu for immediate presentation. The proposed model path consumes synchronized records in an asynchronous service. This preserves a usable engine while the visual path is evaluated. Research on video models' physical generalization motivates the boundary: controlled motion/collision tests have found gaps on unseen physical conditions. Those findings support testing against explicit state, rather than assuming that plausible video establishes physical correctness. [11] [R15]

How recent research informs the design

The recent signed article located in this review discusses GPT-6 Astra, with an authorized republication dated 2026-09-16. Yuanming Hu argues that coding/spatial understanding and detailed 3D generation complement each other: one helps organize scenes and mechanics, while the other supplies asset detail. This is the author's assessment; its model comparisons are not Pajama performance or product acceptance results. It motivates separately integrating and evaluating agent orchestration and asset generation. [R16]

The Meshy team's Mora article (2026-09-09) combines code, 3D control signals and video output. It also discloses early limitations in portal views, character detail and latency. We adopt the division of responsibilities as a design reference, retaining native presentation and separate acceptance gates. A public demonstration does not establish an available compatible SDK for Pajama. [R17]

Other approaches suggest different interface choices:

Pajama's model interfaces should therefore accommodate content proposals, action proposals and generated observations, each with its own lifecycle. Learned state prediction can initially support offline tests, planning or candidate proposals. Future authoritative execution would require explicit state formats, budgets, recovery and consistency protocols. This is a project engineering choice, not a claim that models can only ever render.

Connecting code, assets and visual feedback

The intended loop starts with an agent reading live capabilities and authorized observations, writing bounded Luau mechanics and construction rules, and preparing assets through a specialized 3D service or modeling tool. It then publishes a reviewable Package. After activation, the engine provides queryable results and native frames; the planned observation adapter supplies model presentation and evaluation inputs. The agent uses that feedback to propose another revision. This loop reuses admission and revision checks; its complete integration still requires acceptance. [14] [4] [85]

For a destructible research station, code describes floor layouts, component relationships and skill rules; assets supply appearance, rigs, animation and collision candidates. The server accepts publication and determines construction, hits and destruction. Generated scenes or 3DGS also require supported resource formats, units, collision and budget checks: visual completeness does not establish interactive semantics. Observations bind to activated content and world identities; subsequent agent edits carry their observed revision so stale feedback cannot overwrite newer state. [5] [83] [17]

Three completion criteria remain separate: content is accepted and executes, world outcomes survive recovery, and generated presentation is faithful and usable. An external model upgrade can improve one stage without establishing evidence for the others.

H3 or Wan: choose the adaptation starting point by task

As of 2026-09-21, prioritize a causal, few-step Wan 2.2 TI2V-5B route for adaptation and cost experiments, retain H3 as a conditioning/quality baseline, and include SANA-Streaming in the first presentation comparison. This orders experiments; it is not an engine performance result. With explicit world state already available, the first task is to represent that state faithfully, rather than ask a model to infer collisions and destruction again.

The critical adaptation gap is continuously changing geometry. SolarWM's reported experiments use a fixed scene prompt, with camera trajectories as the time-varying external control. Camera exploration does not establish faithful rendering of a newly built floor, a fresh wall opening or an asset revision. Pajama needs adapters and paired data supporting continuous RGB/geometric conditions, or must restrict a candidate to camera-exploration research. [R26]

Frontier approaches to interaction, memory and cost

These are verifiable approaches relevant to engine integration, not a cross-task leaderboard. External results retain their hardware, measurement scope and release status.

ReWorld and LingBot-World 2.0 currently specify CC BY-NC-SA 4.0. They are research references, not automatic additions to the platform's commercial deployment list. Each actual candidate also requires separate checks of base weights, adapted weights, data and dependencies; a code license does not cover all of them. [R27] [R28]

MiniMax H3 as a conditioning and quality baseline

H3 remains a conditioning and quality baseline, rather than the sole first integration candidate. Its released FL2VA and Ref2VA base checkpoints offer a starting point for conditioned video generation. The official release distinguishes local 768p base output from the full pipeline's hosted Context-IR and, at release, unavailable local Regenerate-2K checkpoint. The initial experiment must pin the exact checkpoint, license, conditioning path and deployment dependencies. The advertised 24 FPS describes output playback, not inference throughput or input response. [R31]

Two recent preprints provide more specific evidence:

These results motivate a reproduction and adaptation experiment; they do not establish an existing Pajama integration. [R32] [R33]

Those prototypes' limitations are not permanent limits of the H3 backbone: subsequent SolarWM-H3 releases provide a causal streaming route. They still do not establish continuous geometry conditioning, multi-client consistency or target deployment cost for Pajama. Selection records must distinguish base H3, H3-World, SolarWM-H3 and hosted H3 Max. [R22]

Pajama would supply the persistent environment, accepted actions and versioned content missing from a standalone clip workflow. The adapter work includes aligned training pairs, supported conditioning channels, model adaptation and evaluation. Sending depth to an unmodified public video endpoint is insufficient unless that interface explicitly supports it.

The visual objective is specified through appearance references and aligned target imagery, with recorded rights and provenance. Engine proxies describe structure and motion; they do not by themselves supply a richer visual style. Experiments should therefore measure both adherence to state and the visual benefit over the native baseline.

The first use case is recorded-world replay. Full recorded trajectories can condition a clip, including information that was future at its first frame. This measures visual conditioning. A separate causal test restricts each generation request to inputs available at its declared decision time. Delayed spectator output must declare its lookahead and display delay; live evaluation must exclude future actions and state from both clip preparation and conditioning.

A state-to-observation interface

The existing boundaries are useful starting points, with distinct timing semantics. PhysicsFrame publishes dynamic/vehicle poses and can return a cached frame with fresh=false. OwnedRenderSnapshot captures presentation inputs and an extraction generation, without an authoritative tick/epoch identity. The bridge must reconcile authoritative snapshots, actual physics publications and camera/presentation sampling; concatenating these structures would not establish alignment. [15] [16] [85]

Each inference session receives only its authorized view. Privileged reference state may be retained separately for evaluation. An observation exporter must preserve player/spectator visibility and interest rules so hidden entities and internal state do not leak through model conditions.

A feasible first deliverable is a bounded recorder and offline reader: an episode manifest, timestamped state/action/event records, and aligned reference images. An offline or headless renderer can generate additional channels without placing inference in the tick. Queue pressure, dropped samples and timing gaps are recorded explicitly. Depth and semantic-ID adaptation follow once paired capture is validated; other channels remain capabilities to negotiate, not assumptions about a candidate's inputs. [69] [85]

Observations must also distinguish capture progress from durability progress. Planned records carry the sampled tick, authority lifetime and the known durable checkpoint's identity and coverage. A visible effect is not automatically part of the recovery root, and a missing durability acknowledgement does not establish a failed write. Capture must not block simulation waiting for a checkpoint. Recovery starts a new episode and model session from the recovered state. Reconcile the historical tail against the checkpoint actually restored, marking records as included, outside that recovery root or still unknown. Observation records cannot become an alternate world save. [6] [85]

Admitting model output

The proposed adapter binds each request to its episode/session, authority lifetime, content identity, authorized-view revision, camera conditions and input range. Output is eligible only while the receiver accepts those conditions and its declared display window remains valid. Content changes, recovery, view-permission changes or incompatible camera discontinuities require checks on pending work and cached history; ordinary camera motion follows the declared condition sequence. Cancellation does not prove that a background job has stopped, so late results must pass the same admission check.

This checks provenance and freshness; agreement between pixels and geometry still requires separate evaluation. The first test can use a mock adapter that returns delayed, reordered results, proving that obsolete output cannot enter presentation and native fallback remains available before connecting real model inference. [85]

Quality and operating efficiency

Evaluation asks whether output preserves the visible facts of an interaction, and what that costs. Physion provides a precedent for scenario-based tests of physical events; VBench separates temporal, motion and appearance dimensions of video quality. Pajama needs both kinds of measurement alongside its engine reference state: neither benchmark alone establishes interactive game correctness. [R34] [R35]

Use native reference rendering as shared truth, comparing supported Wan/H3 conditioning and SANA video transformation with their input capabilities and application scopes declared separately. Keep scenes and trajectories separate between training and evaluation, avoiding adjacent-frame leakage. Use held-out interactions and multiple seeds; publish sample counts, failure rates and uncertainty. Geometric scoring needs visibility masks and calibrated extraction, with manual review where a stylized image makes automatic measurement ambiguous. Set acceptance thresholds for the intended workload before running the comparison.

Shared entity IDs and correct input receipts establish protocol identity; they do not prove that generated pixels agree. Multi-view geometry, appearance and event agreement remain measured research objectives. The inference worker has no mutable world handle. Timeouts, wrong epochs and stale sessions can trigger deterministic rejection and a native-renderer fallback; reliable automatic detection of visual mismatch requires additional research. Fallback preserves availability and authoritative play, while transition quality must also be tested. Precision-sensitive targeting and UI remain engine-rendered in the initial design. [85]

This work extends the engine's established design choices: explicit versions, bounded interfaces and one simulation authority per world. Native release readiness, content continuity and distributed operation remain the foundation for the model path. [1] [26] [86] [53] [67] [27]

Inference cost: reduce the task before optimizing computation

Optimize in this order, rechecking state adherence and visual benefit at each step:

1. Reuse known engine results. Produce inexpensive reference views and supported geometry conditions from actual state; test presentation transformation first. Native systems provide collision, destruction, aiming and UI. Assets and static appearance reusable through Packages need not be generated anew for every player frame.
2. Reduce online work. Compare smaller backbones, lower working resolution and short generation chunks; reserve expensive refinement for replay or other non-immediate uses. Report resolution, compression and chunk size together: larger chunks can improve throughput while delaying input collection and first output. Reprojection of slower model output is a display strategy, not newly generated frames or a remedy for stale state.
3. Use causal few-step distillation and bounded memory. Start from released distilled weights to reduce denoising work; adding LoRA alone does not reduce backbone inference computation. Retain necessary history and retrieve by space and revision; recovery, visibility changes, destruction and content switches invalidate it. Measure step-reduction gains across the full pipeline.
4. Optimize the whole generation pipeline. Profile condition encoding, DiT, VAE, transport and display before selecting hardware-compatible precision, fused kernels, compilation and caches. SANA-Streaming's end-to-end versus DiT rates illustrate why denoiser optimization alone is insufficient. Recheck small structures, motion boundaries and event timing after quantization. [R25]
5. Schedule against latency targets. Separate cold starts from warm requests; bound batching, concurrency and queues by deadlines, and cancel obsolete work. Warm pools trade startup delay for idle expense. Different cameras or visibility permissions cannot simply share generated frames; shared assets or encodings still require matching identity and access contracts.

Account for cost per minute of valid, timely delivered model output, accumulating minutes per receiving view. Also report total cost per active player-minute and model coverage:

Cost per valid output minute = (all GPU billing + API + transport and other serving costs) / valid, timely delivered model-view minutes

GPU billing includes warm idle capacity, failures, retries and work that continues after cancellation. Do not double-count a charge as both API and self-hosted GPU expense. Exclude stale, rejected, duplicate and interpolation-filled segments from the denominator; record native fallback separately. A run with no valid output fails, rather than reporting zero cost. Report training/adaptation separately and state amortization assumptions against expected usage.

A shared-scene experiment to decide which models proceed

Fix a destructible-building trace covering turns, occlusion, construction, wall damage, departure and return, two cameras and content revision changes. Validate capture and a mock adapter before comparing causal Wan, conditioned/causal H3 and SANA video transformation. Candidates unable to represent live changes enter only the offline group, not the online ranking.

Retain engine/weight/adapter versions, input channels, resolution, precision, denoising steps, chunk length, cache budget, GPU type/count, concurrency and complete timestamps. Report file playback FPS, fresh generation FPS including codecs, time to first frame, and input-to-actual-display latency separately. The last includes capture, chunk assembly, queuing, network and presentation. Runs supplied with a whole future trajectory establish offline capability only.

Apply declared state-consistency and response gates before comparing valid-output cost. Separate single-view and concurrent-view workloads; verify native availability under timeouts, memory pressure and late output. This review establishes sources and experiment design, not Pajama H3/Wan/SANA inference measurements. No per-user price, real-time frame rate or proven winning model is claimed.

17 · Development roadmap

Two tracks, one foundation

Development advances along two connected tracks. The runtime track makes a persistent world easier to create, operate and extend. The model track turns that world's state into controllable generated presentation. Both build on the same identities, content versions and recovery contracts. [19] [27]

Next work, ordered by the new evidence

These priorities refine M1–M5; they do not replace the single-authority product goal or promise delivery dates. Expanding one qualified authority comes before speculative sharding. Model presentation remains M6 and cannot bypass these world contracts. Architecture assessment.

M1 · Playable content packages

Consolidate demonstrated player actions, Package behavior and matching presentation into the complete release workflow. Terrain/checkpoint evidence and the construction/breach rehearsal provide the starting points. The integration gate combines request handling, durable replay/cooldown, exact-version cue playback, update and recovery in one run. [24] [25] [21]

Completion criterion: two native clients observe the same accepted action and effect through update, rollback and recovery. Repeated requests produce one world result. This becomes the reference workflow for adding further gameplay capabilities.

M2 · Content continuity

Extend resource retention from the active and immediate rollback versions to the lifetime requirements of persistent instances. Define which instances retain their original version, which may migrate, and how historical resources are retained and reclaimed. [26] [23]

Completion criterion: retained instances resolve the correct content after updates and restarts; publication and cleanup preserve every required reference.

M3 · Native release readiness

Bring gameplay and content continuity into the normal client/server startup path. Complete catalog ordering, compatibility handling, platform validation and workload-based performance measurements. [35] [86] [19]

Completion criterion: the normal application flow passes the two-client publication, update, reconnect, restart and rollback scenario, including missing or corrupt resources. This establishes a repeatable native release baseline.

M4 · Persistent world operations

Build on the leased authority and checkpoint foundation to connect player identity, world discovery, admission and recovery. Complete persistent-write fencing and the profile-effect outbox, then exercise replacement of a failed authority. [7] [53] [40]

Completion criterion: a replacement authority restores a coherent world and admits reconnecting players while rejecting late writes from the previous authority. Capacity measurements guide subsequent scaling decisions.

World expansion: acceptance across M3 and M4

Implement the spatial contracts under one authority before deciding whether cross-server partitioning is needed. All stages below are new acceptance work still to complete:

These gates separately establish spatial expansion, persistence and visual continuity. Until they pass, demonstrations should identify their current map extent. Expansion remains subject to address-space limits, storage quotas and operational capacity; it does not promise to load or permanently retain infinite resources.

M5 · Human and agent creation

Build on the implemented prepared-candidate workflow: add isolated preview, persistent job/cancellation budgets, stable third-party contracts and an author-to-player handoff. Exact-digest publication, lost-response reconciliation and conflicting-generation rejection already have local evidence. Generation services supply candidate assets and behavior through the same admission process. [66] [67] [1]

Completion criterion: an author can introduce a playable capability, inspect its result and manage its lifecycle through one documented workflow; receipts correlate publication stages, sustained generation stays within aggregate budgets, and lost responses remain reconcilable.

M6 · Model-generated world presentation

Develop a provider-neutral observation and presentation layer, prioritizing Wan adaptation, H3 quality comparisons and SANA video transformation experiments. Synchronized capture can advance alongside core-runtime integration; production presentation depends on reliable content and recovery lifecycles. Distributed model sessions build on M4. [3] [85]

The initial scenario set covers a falling body reaching rest, a character returning from occlusion, a persistent terrain or wall opening, and two cameras observing one interaction. These exercises connect visual quality to the facts that make a world playable. [27]

The planned executable evidence is an observation round-trip validator, an offline comparison runner and a session fault-injection suite, extending the native lifecycle runner. A replay result permits a replay application. Spectator and live applications require their own causal, latency and consistency gates. If an adapter misses them, it remains a replay/creative-output option while the native game path continues to develop.

Validation across the lifecycle

Each milestone is reviewed through its delivered capability and reproducible integration results. The existing native package runner provides the lifecycle baseline; planned capture, model-comparison and session-failure tooling extends it with synchronized observations and real-model measurements. The path progresses from replay to spectator presentation, then toward live output as control quality, latency and operating efficiency meet the needs of the experience. [84] [85]

18 · Terminology and reading guide

For a project overview, read vision → architecture → progress → roadmap. For the architecture, read execution flows → world data → publication → recovery. For authoring or client integration, continue through delivery → rendering → authoring → worked example. The references below identify the source and design records behind the implementation statements.

19 · References and evidence

Implementation evidence and planning documents are pinned to their respective source revisions. Section 15 explains their scope; external research and its design relevance are listed separately below.

Implementation references use repository paths and line numbers. Repository access is managed separately.

1. Product goal and durable boundarydocs/PRODUCT_GOAL.md:10 @ 026acf9
2. Native client dependenciescrates/pajama-client/Cargo.toml:21 @ 026acf9
3. Planned direction: world-model interoperabilitydocs/PRODUCT_GOAL.md:28 @ bf13e81
4. Permanent package formats and identitydocs/CONTENT_PACKAGES.md:15 @ 026acf9
5. Realm-owned package admissiondocs/CONTENT_PACKAGES.md:595 @ 026acf9
6. Composite checkpoint coordinatorcrates/pajama-server/src/authority_checkpoint_coordinator.rs:1 @ 026acf9
7. Backend scope and production gatesdocs/BACKEND_ARCHITECTURE.md:3 @ 026acf9
8. Real authoritative tick orderingcrates/pajama-server/src/lib.rs:4025 @ 026acf9
9. Atomic client presentation coordinatorcrates/pajama-client/src/content_presentation.rs:1 @ 026acf9
10. Bounded client delivery ownercrates/pajama-client/src/content_runtime.rs:1 @ 026acf9
11. Render layers and execution ownershipdocs/RENDERING_ARCHITECTURE.md:41 @ 026acf9
12. Workspace ownership mapCargo.toml:3 @ 026acf9
13. Bounded package archive stagingcrates/pajama-content/src/lib.rs:323 @ 026acf9
14. Immutable frames, typed commands and budgetscrates/pajama-script/README.md:17 @ 026acf9
15. Solver interfacecrates/pajama-physics/src/lib.rs:297 @ 026acf9
16. RenderSnapshot boundarycrates/pajama-render/src/snapshot.rs:246 @ 026acf9
17. Package publication servicecrates/pajama-server/src/package_publication.rs:1 @ 026acf9
18. Interest-managed replicationcrates/pajama-server/src/replication.rs:1 @ 026acf9
19. Completion gate and implementation frontierdocs/ROADMAP.md:8 @ 026acf9
20. Native observers and live Package construction rehearsalscripts/authoring/building-rehearsal.mjs:1 @ fe4d401
21. Player-intent breach test, authoritative damage and replicated removalscripts/authoring/building-probe.mjs:1 @ fe4d401
22. WorldDoc persistence schemacrates/pajama-world/src/lib.rs:489 @ 026acf9
23. Recorded native update, rollback and recovery evidencedocs/ROADMAP.md:294 @ 026acf9
24. Recorded durable package terrain evidencedocs/ROADMAP.md:371 @ 026acf9
25. Skill binding and remaining playable gatedocs/ROADMAP.md:419 @ 026acf9
26. Historical instance policy and V5 boundarydocs/ROADMAP.md:437 @ 026acf9
27. Planned P6 stages and evidence gatesdocs/ROADMAP.md:476 @ bf13e81
28. Authority lifecycle and fencingcrates/pajama-server/src/authority.rs:24 @ 026acf9
29. Verified content-addressed cachecrates/pajama-client/src/content_cache.rs:1 @ 026acf9
30. Read-only defaults and explicit write scopescrates/pajama-mcp/README.md:32 @ 026acf9
31. Entity documentcrates/pajama-world/src/lib.rs:384 @ 026acf9
32. Runtime world and change trackingcrates/pajama-world/src/lib.rs:533 @ 026acf9
33. World mutation stampcrates/pajama-world/src/lib.rs:796 @ 026acf9
34. Standalone ECS dependencycrates/pajama-sim/Cargo.toml:10 @ 026acf9
35. Catalog ordering evidence and open gatesdocs/ROADMAP.md:173 @ 026acf9
36. Local V1 input and composite publicationdocs/ROADMAP.md:104 @ 026acf9
37. Closed checkpoint backend ownershipcrates/pajama-server/src/authority_checkpoint_backend.rs:1 @ 026acf9
38. Local immutable store and head CAScrates/pajama-server/src/local_checkpoint_store.rs:1 @ 026acf9
39. Process-local checkpoint lifetimecrates/pajama-server/src/process_local_checkpoint.rs:1 @ 026acf9
40. Rollback integration and production boundarydocs/ROADMAP.md:135 @ 026acf9
41. Generation-bound CPU decodercrates/pajama-client/src/content_decode.rs:1 @ 026acf9
42. Imported GPU resource generationscrates/pajama-render/src/skinned/asset_generation.rs:1 @ 026acf9
43. Exact asset extraction and placeholderscrates/pajama-client/src/package_asset_presentation.rs:1 @ 026acf9
44. Rapier default and Box3D compatibilitycrates/pajama-physics/src/lib.rs:402 @ 026acf9
45. Fixed substeps and bounded catch-upcrates/pajama-physics/src/rapier_backend.rs:60 @ 026acf9
46. Authoritative causal terrain editcrates/pajama-server/src/terrain_sync.rs:232 @ 026acf9
47. Client movement prediction and echo reconciliationcrates/pajama-client/src/app_runtime.rs:541 @ 026acf9
48. Finite terrain domain contractcrates/pajama-world/src/terrain.rs:35 @ d96cddf
49. Default terrain half-size: 250 metrescrates/pajama-world/src/terrain.rs:310 @ d96cddf
50. Accepted finite extent rangecrates/pajama-world/src/terrain.rs:362 @ d96cddf
51. Authoritative movement clamps to terrain extentcrates/pajama-server/src/lib.rs:4556 @ d96cddf
52. Extent-bounded physics ground tile schedulingcrates/pajama-server/src/ground_schedule.rs:214 @ d96cddf
53. Distributed authority roadmapdocs/ROADMAP.md:452 @ 026acf9
54. Compiled render pipelinecrates/pajama-render/src/pipeline.rs:389 @ 026acf9
55. Animation architecturedocs/ANIMATION.md:1 @ 026acf9
56. Four-client render profile regression and controlled measurementsdocs/evidence/four-client-render-profile/README.md:1 @ e2134b8
57. Spectral water and consistent cloud radiancedocs/TIDEWATER_ROUND3.md:1 @ e2134b8
58. Lake transport and oblique wave filteringdocs/LAKE_REALISM.md:1 @ e2134b8
59. Shoreline foam implementation and acceptancedocs/SHORE_FOAM.md:1 @ e2134b8
60. Cloud detail footprint filtering and evidencedocs/CLOUD_DISTANCE.md:1 @ e2134b8
61. Native weather implementation and limitsdocs/WEATHER_SYSTEM.md:1 @ e2134b8
62. Underwater optics numerical and native acceptancedocs/evidence/underwater-optics/README.md:1 @ e2134b8
63. Native 40 km terrain horizon and streaming acceptancedocs/evidence/terrain-horizon-lod/README.md:1 @ e2134b8
64. Command/component registrycrates/pajama-world/src/registry.rs:1 @ 026acf9
65. Live registry discovery and authorizationcrates/pajama-mcp/README.md:3 @ 026acf9
66. Existing audited script runnercrates/pajama-mcp/README.md:61 @ 026acf9
67. Package authoring and generation roadmapdocs/ROADMAP.md:465 @ 026acf9
68. At-most-once send and uncertain outcomescrates/pajama-mcp/README.md:52 @ 026acf9
69. Native headless rendering pathcrates/pajama-client/src/headless.rs:951 @ 026acf9
70. Jev integration scope, controls and acceptancedocs/SYSTEM_ONE_AGENT.md:7 @ 382ca36
71. Approved skill choice, budget, freshness and receipt policyscripts/system-one/skill-director.mjs:35 @ 382ca36
72. Ordinary player observation and authoritative skill receiptsscripts/system-one/game-client.mjs:46 @ 382ca36
73. Authoring tools, Codex client acceptance limit and remote integrationdocs/AGENT_AUTHORING_DEMO.md:118 @ 382ca36
74. Real Jev decisions and bilateral authoritative terrain evidencedocs/evidence/jev-system-one-2026-09-21.json:82 @ 382ca36
75. Jev native Decisions transport and version validationscripts/system-one/jev.mjs:98 @ 382ca36
76. Explicit authoring MCP adapter to the existing runtimescripts/authoring/mcp.mjs:1 @ 382ca36
77. Durable Cloudflare job queue, bounded tools and claimsservices/pajama-author-agent/src/index.ts:31 @ 382ca36
78. Proposed engine evolution control plane, contracts and acceptance gatesdocs/ENGINE_EVOLUTION.md:7 @ bb964ad
79. Engineering decision roles, conditional practices and proposed enforcementdocs/ENGINE_DECISIONS.md:9 @ e1dc2c0
80. Repository agent routing to the engineering decision guideAGENTS.md:33 @ e1dc2c0
81. Existing controlled performance comparison and retained verdictsdocs/PERFORMANCE_LOOP.md:156 @ bb964ad
82. Engine correctness CI and stable-runner performance boundary.github/workflows/ci.yml:10 @ bb964ad
83. Versioned building kit, procedural behavior and player skill bindingscripts/authoring/complex-building.mjs:7 @ fe4d401
84. Native two-client test runnerservices/pajama-edge/test/native_catalog_v2_two_client_e2e.mjs:9 @ 026acf9
85. Planned observation, provider and presentation contractsdocs/WORLD_MODEL_BRIDGE.md:8 @ bf13e81
86. Platform cleanup and CI scopedocs/ROADMAP.md:70 @ 026acf9

External research and design references

These publications inform the proposed design. Their results describe the cited systems, not capabilities already demonstrated by Pajama Engine.

References checked: 2026-09-21.

1. [R1] World ModelsDavid Ha and Jürgen Schmidhuber · 2018 · Research paper

   Learns compressed spatial and temporal representations of environments for prediction and policy training. It grounds the distinction between a learned environment model and its execution environment; a physics engine or observation export alone does not constitute a trained world model.

2. [R2] Networking OverviewEpic Games · Accessed 2026 · Official engine documentation

   Documents the separation between authoritative server gameplay, replicated client state, and visual presentation. This established multiplayer pattern informs Pajama's authority boundary; it does not validate Pajama's implementation, network performance, or a learned renderer's consistency.

3. [R3] Streaming 3DGS worlds on the webWorld Labs · 2026-04-14 · Official renderer engineering article

   Explains view-dependent LOD, progressive streaming and fixed-pool GPU paging for 3DGS. Informs bounded residency, without supplying destructible terrain, authoritative collision or persistence.

4. [R4] Extract in bevy::renderBevy contributors · Accessed 2026 · Bevy 0.19.1 API documentation

   Documents read-only extraction from the simulation world into a separate render world, allowing simulation and rendering work to overlap. This is an engineering precedent for observation boundaries; it does not establish serialized replay, model conditioning, or cross-machine determinism.

5. [R5] Model Context Protocol — ToolsModel Context Protocol contributors · 2025 · Protocol specification, 2025-06-18

   Defines discoverable tools, JSON Schema inputs, optional output schemas, structured results, and server validation requirements. It supports a typed authoring interface; domain authorization, resource budgets, revision checks, and atomic world activation remain responsibilities of the engine implementation.

6. [R6] Embedding a sandboxed Luau virtual machineLuau contributors · Accessed 2026 · Official language documentation

   Describes restricted libraries, protected environments, and host-controlled interrupts for sandboxing. It informs bounded script execution; embedders must configure memory limits and exposed APIs, and the documented sandbox is not a formal proof of isolation or safety for arbitrary host bindings.

7. [R7] System One introductionTypeSafe AI · 2026 · Official model documentation

   Describes typed decision models. Supports the narrow decision role; it does not establish Pajama latency, autonomy or full visual understanding.

8. [R8] ConfidenceTypeSafe AI · 2026 · Official confidence documentation

   Explains confidence and decision distributions. Thresholds need task evaluation; confidence is neither permission nor proof of correctness.

9. [R9] Jev 1.13 APIOpenRouter · 2026 · Official provider API documentation

   Documents Jev availability through OpenRouter. The engine adapter pins the native Decisions contract and verifies the actual returned version; measured costs belong to the linked run receipt.

10. [R10] Model Context ProtocolOpenAI · 2026 · Official Codex MCP documentation

    Documents config.toml MCP entries, stdio command/args/cwd/environment and timeouts. Correct configuration does not establish successful Pajama publication or bypass client approval.

11. [R11] The Last AI Built by Humans: Toward Genuine Recursive Self-ImprovementYi Duan et al. · 2026-09-15 (v2) · Survey preprint

    Motivates distinguishing artifact gains from inherited improvements to the improver. Pajama's control-plane design is a proposal, with no autonomous-evolution result claimed.

12. [R12] Introduction to Performance Profiling and ConfigurationEpic Games · 2026 (accessed) · Official engine documentation

    Connects frame timing and performance budgets to target hardware. It informs measurement practice, not Pajama performance claims.

13. [R13] Optimizing CPU-side Rendering CodeClay John / Godot Engine · 2026-09-15 · Official engine engineering report

    Demonstrates profiling, diagnosis and remeasurement, including counterexamples to intuitive caching optimizations. Reported gains belong to Godot's test scenes.

14. [R14] Canarying ReleasesAlec Warner and Štěpán Davidovič / Google SRE · 2018 · Primary release-engineering guidance

    Supports limited rollout with control comparisons, representative samples and absolute health measures. Pajama must apply this without concurrent writers in one world.

15. [R15] How Far is Video Generation from World Model: A Physical Law PerspectiveBingyi Kang et al. · 2025 · ICML 2025 research paper

    Finds out-of-distribution physical generalization failures in controlled two-dimensional video prediction experiments. It motivates testing against explicit physical state and unseen conditions; its findings concern the tested models and setup, rather than proving a universal limitation of video generation.

16. [R16] GPT-6 Astra and the value of 3D generationYuanming Hu (Ethan); authorized republication via Founder Park / BAAI · 2026-09-16 · Signed commentary, authorized republication

    Argues for complementary coding/spatial understanding and detailed asset generation. This is the author’s assessment, not an independent model benchmark or Pajama acceptance result.

17. [R17] Mora: Beyond the “World Model”Meshy Research Team · 2026-09-09 · Official architecture and prototype report

    Combines coding, 3D control signals and generated audiovisual output; discloses early visual and latency limitations. It motivates modular integration without establishing a compatible Pajama SDK.

18. [R18] A Functional Taxonomy of World ModelsWorld Labs · 2026-06-03 · Official research perspective

    Classifies renderers, simulators and planners by observation, state and action outputs, while discussing convergence. Informs output-specific contracts; generated geometry still needs validation.

19. [R19] Atlas: A World Model for Spatial IntelligenceWorld Labs · 2026-09-01 · Official model announcement

    Describes spatially grounded multimodal context, camera control and explicit 3D outputs. The early-access announcement motivates interface research, not a claim of Pajama integration or measured performance.

20. [R20] Project Genie: Experimenting with infinite, interactive worldsDiego Rivas, Elliott Breece and Suz Chambers (Google) · 2026-01-29 · Official prototype launch report

    Reports action-responsive exploration and the launch prototype’s physics, control and 60-second generation limits. Supports separating exploration from durable world acceptance; these are dated prototype limits.

21. [R21] Wan2.2: Open and Advanced Large-Scale Video Generative ModelsWan team · Accessed 2026-09-21 · Official model repository

    Documents the 5B base and offloaded 24 GB inference. This is a feasibility starting point, not evidence for interactive latency or a continuous state-conditioned adapter.

22. [R22] SolarWM: release status and model familyJunchao Huang et al. · 2026-09-13; checked 2026-09-21 · Official research repository

    Records three-stage Wan 5B and H3 releases, including September 13 streaming H3. Release availability does not establish live engine integration.

23. [R23] SolarWM Wan2.2 TI2V-5B backend guideSolarWM contributors · Accessed 2026-09-21 · Official inference guide

    Provides a single-GPU Stage2 inference example. An example launch is neither a hardware minimum nor a same-workload cost comparison.

24. [R24] SolarWM MiniMax-H3 backend guideSolarWM contributors · Accessed 2026-09-21 · Official inference guide

    Documents H3 stage checkpoints and eight-GPU inference examples. No eightfold cost ratio or minimum configuration is inferred.

25. [R25] SANA-Streaming: Real-time Streaming Video Editing with Hybrid Diffusion TransformerYuyang Zhao et al. · 2026-05; released 2026-06 · Research preprint, v1

    Studies source-video-conditioned streaming editing and separates whole-pipeline from denoiser throughput. Engine geometry fidelity and input-response latency require separate tests.

26. [R26] SolarWM: Open Data and Scalable Training for Long-Horizon Video World ModelsJunchao Huang et al. · 2026-09-02 · Research preprint, v1

    Describes causal adaptation and distillation across backbones. Section 7.1 uses fixed prompts and time-varying camera control, not arbitrary live geometry updates.

27. [R27] ReWorld: An Interactive World Model with Long-Horizon MemoryReWorld authors · 2026-08/09; checked 2026-09-21 · Official research repository

    Documents bounded caches, landmark retrieval and few-step generation. The current repository specifies non-commercial terms; model memory does not replace versioned engine persistence.

28. [R28] LingBot-World 2.0: Infinite Worlds with Versatile InteractionsRobbyant / LingBot-World team · 2026-07; checked 2026-09-21 · Official model repository

    Provides causal model variants and multi-GPU examples. Non-commercial terms and unreleased deployment code constrain reuse; headline throughput is not single-player latency.

29. [R29] SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion TransformerHaoyi Zhu et al. · 2026-05-14 · Research preprint, v1

    Studies efficient camera-conditioned generation with hybrid attention. The RTX 5090 minute-video result measures denoising, not input-to-display latency.

30. [R30] H3 Max: Built with fal Inference and Trainingfal · 2026-09-17 · Provider engineering report

    Describes fast clip generation and multi-node serving with startup optimizations. Provider-reported speed does not establish continuous control or per-player cost.

31. [R31] Open General Intelligence: MiniMax H3 Is Now Open SourceMiniMax · 2026 · Official model release documentation

    Describes released FL2VA and Ref2VA base checkpoints and multimodal conditioning. The documented full 2K workflow also uses hosted components; its 24 FPS output specification is a playback rate, not a measurement of generation throughput or interactive latency.

32. [R32] H3-World: Turning Language Understanding into World ControlDanze Chen et al. · 2026 · Research preprint

    Adapts H3 using temporally aligned language instructions for character and camera control. The study supports short-horizon controllability; it explicitly leaves persistent state, real-time interaction, planning, and policy learning outside the demonstrated capabilities and calls for broader reliability evaluation.

33. [R33] Code World Model: Coding Agent as World BrainYiwen Chen, Guosheng Lin, and Chi Zhang · 2026 · Research preprint

    Separates executable world evolution from video synthesis and adapts H3 Ref2VA to depth and semantic proxy conditions. Qualitative results support investigating this interface; the prototype omits autoregressive real-time generation, and its control comparisons exclude inference latency.

34. [R34] Physion: Evaluating Physical Prediction from Vision in Humans and MachinesDaniel M. Bear et al. · 2021 · Research paper and benchmark

    Introduces reproducible physical prediction scenarios covering contact, stability, rolling, and other dynamics. It motivates scenario-based physical evaluation; it evaluates prediction from observations, so generated game footage still needs separate tests against authoritative geometry, event timing, and gameplay state.

35. [R35] VBench: Comprehensive Benchmark Suite for Video Generative ModelsZiqi Huang et al. · 2023 · Research paper and benchmark

    Separates video quality into dimensions including identity consistency, motion smoothness, flicker, and spatial relationships. It informs visual evaluation alongside state-based checks; high perceptual scores alone do not establish collision accuracy, event fidelity, multiplayer agreement, or acceptable interaction latency.