Predictive Systems

[3mcd]

Overview

I recently helped port the fantastic prediction/reconciliation algorithm CrystalOrb from Rust to JavaScript. The JS port works very well (see a very rough demo here) but CrystalOrb's current design (see this issue) works best when most game logic is located in the World.step function. Furthermore, the CrystalOrb client maintains two separate (but complete) simulations, without exposing one as the source of truth for client-only physics (e.g. particle physics).

This design makes it a bit tricky to implement ECS alongside CrystalOrb, since ECS logic is traditionally implemented as small, pure-ish functions (in that they usually only modify a small subset of entities and components), and writing non-predictive code is a bit difficult without implementing an entire command/snapshot/interpolate workflow, or writing similar command logic around the CrystalOrb "protocol".

It would be awesome if Harmony (and eventually Javelin) had support for this kind of predictive algorithm out-of-the-box. I'm dubbing this "Predictive Systems" in the meantime. Of course, currently Harmony doesn't have a concept of systems at all (only entities, components, and queries), but new structures could be introduced with this proposal which cover other best practices for system design and organization outside of netcode.

Proposal

Out-of-scope

• Peer-to-peer. I think designing the structures and functions involved within the model of the classical authoritative server <> client architecture is necessary to keep goals reasonable.
• Network transport/message protocol. User (or consuming library i.e. javelin) will need to bring their own encoder/decoder and networking stack.

In-scope

CrystalOrb does a few things that we can learn from:

• Clock synchronization of client and server using timestamps (although the author has some regrets about using wrapped i16 vs float64 -- I think the wrapped approach is cool and shaves a few bytes off each snapshot message)
• Client prediction with fast-forwarding using multiple simulations
• Commands
• Interpolation
• Client can tick at monitor refresh rate and still be accurate

Things we'll want to add

• Easily distinguish between predictive systems and states (components) and non-predictive ones
• Abstract the "current" predicted state to non-predictive systems (i.e. it seems like there is only one physics world to the client)
• Allow predictive state and non-predictive state to easily mesh

New structures and functions to consider

• Timestamps
• ClockSync
• Command
• Systems vs. predictive systems, auto-creating Archetype columns for predicted state, snapshot state, and maybe interpolated state?

API Sample

// System is defined like normal, without any notion of prediction
function input(ecs) {
    let command: InputCommand
    while (command = Command.take(ecs, CommandType.Input)) {
        const velocity = Entity.get(ecs, command.entity, Velocity)
        if (command.jump) {
          velocity.y += 10
        }
        // ...
        Entity.set(ecs, command.entity, Velocity, velocity)
    }
}

// On the client, we define a blend function that could interpolate between
// the latest predicted state and previous snapshot
function blend(queryFrom, queryTo, t) {

}

const pipeline = Pipeline.make(
    // On the client, a predictive system is configured with one or more
    // types to simulate in the future (relative to a server). Queries executed
    // within the predictive systems would somehow interact with "overlayed"
    // entity tables that shadow the original archetypes 
    System.makePredictive(input, [Prefabs.Player], [Commands.Input], blend),
    System.make(render),
)

// Some built-in commands would provide clock sync instructions and state
// snapshots to the pipeline's state for each predictive system
const commands = Command.makeCommandBuffer()

Pipeline.step(pipeline, world, commands)

// The server will need to generate snapshots at regular intervals and 
// send timestamped payloads to the client

[3mcd]

Leaving some more scratch work here. The general idea is that a system can be invoked within a context, like a predictive context. Most of the Harmony API would be updated to accept an optional context id, changing the final target of that call. Components of a different context could be stored in parallel with the components today. Contexts would just need to track this mapping from the base context schema id to the corresponding schema id within the context.

import * as Rapier from "@dimforge/rapier3d-compat"
import { Command, Context, Effect, Entity, Format, Schema, World } from "harmony-ecs"

const GRAVITY = { x: 0, y: -9.81, z: 0 }

const RigidBody = Schema.define()
const PlayerInput = Schema.defineBinary(Format.uint8)
const Players = [RigidBody] as const

enum PlayerInputIntent {
  Jump = 1 << 0
}

enum MyCommand {
  Spawn = Command.define({ entity: Format.entity }),
  Input = Command.define({ entity: Format.entity, input: Format.uint8 })
}

enum MyEffect {
  Physics = Effect.defineRef(() => new Rapier.World(GRAVITY))
}

enum MyContext {
  Predicted
}

function simulate(ecs: World.World, ctx: Context.Id) {
  const physics = Effect.call(ecs, MyEffect.Physics, ctx)
  const players = Effect.call(ecs, Effect.Query, ctx)(Players)

  for (const spawn of Command.drain(ecs, MyCommand.Spawn, ctx)) {
    const actor = makePlayerActor()
    Entity.set(ecs, input.entity, [RigidBody], [actor], ctx)
    simulation.add(actor)
  }

  for (const input of Command.drain(ecs, MyCommand.Input, ctx)) {
    Entity.set(ecs, input.entity, [PlayerInput], [input], ctx)
  }

  for (const [e, [b, i]] of players) {
    for (let _ = 0; _ < e.length; _++) {
      const body = b[_] as Rapier.RigidBody
      const input = i[_]
      if ((input | PlayerInputIntent.Jump) === PlayerInputIntent.Jump) {
        body.applyForce(/* ... */)
      }
    }
  }
}

const ecs = World.make(1e5)
const pipeline = Pipeline.make(
  System.make(simulate),
  System.make(simulate, MyContext.Predicted)
)

We could then target one system vs the other with a server snapshot etc.

[3mcd]

Better yet, systems could be passed a World wrapped with the present context id, i.e. a tuple of (World, Context.Id), removing the need for the API to accept the optional context parameter:

// `ecs` is actually (World.World, Context.Id)
function simulate(ecs: World.World) {
  for (const spawn of Command.drain(ecs, Commands.Spawn)) {
    // ...
  }
}