Abstract

This ADR describes the definition of an Entity, how interacts with runtime parts, the implications in memory terms, and also the synchronization between different parts.

What is an Entity?

In the ECS paradigm, the entity is an unequivocal identifier, a number, or a value that represents something abstract. All components referenced by that entity give the meaning of that entity.

The scope of an entity is its own scene, this means that entity 0x1001 in a Scene-A and entity 0x1001 in Scene-B are not linked to each other.

Entity history

This is more related to CRDT ADR, but it's necessary to define it. The entity is the reference to different components. When we create for example a box shape with collision, this has three components (the SDK7 case): MeshRenderer, MeshCollider, and Transform. This collection of components represents a box shape with collision. When we change the property of each component, we are changing the internal state of CRDT.

When we make a remove component operation in that entity, we need to know if in the future I try to add the component again. If we do the operation with every component, at the same time, we probably are giving it the meaning that is completely another entity for us. But the state that something happened before is important to communicate that is completely different.

For example:

// t=t0
// EntityA with Transform (position=[1,2,0]) and MeshRenderer ( $case = 'box')

// t=t1=238442
// There isn't any component with the EntityA

// t=t2=238443
// EntityA with Transform (position=[13,23, 11]) and GltfContainer(src='banana.glb')

In t=t2, we link a Transform to EntityA again, but it's a new version of the Transform that means something different, we know this, because we have the history about the Transform in t=t0, (because of the lamport timestamp)

Reusability approach

The reusability of each reserved but unused entity is important to avoid memory leaks. It's not practical having all entities' history. To illustrate this, we can take two scenes, once a static that only creates its entities once, and then nothing happens, and another one that creates and destroys entities with a balance that MUST be zero (otherwise, it's violating the limits).

The static scene has no problem with the versioning entity, it only creates all the entities that need. But in the dynamic one, with no version, the entity number is always incrementing and this could be a problem, a memory problem. With the preposition of the balance as ZERO (creation-rate - destruction-rate = 0), we can set any number to know how fast the problem appears.

For example, if we have 10,000 entities created per frame, and a 30fps game loop, we'd need almost 4 hours to fill all entities' numbers (without taking into account the entities' versions). Although we always have the existence of 20,000 entities, we have a history of about 2^32 (4,294,967,296) entities.

The extreme previous case can be transformed into a typical case if we take the versioning approach. With the previous rates, we know there are 10k new versions of entities each frame, so each second an entity is being reused 30 times, this is its counter version is incrementing by 30/s. In four hours, the counter is going to be incremented to around 432,000. This number is 10,000 times lower than the first case, which means it would take 100 days to drain the first 10k number of entities.

To summarize, the trick is in the version number. We want to reuse the entity number because it has a block of memory in our state, and using the version number allows us to recycle that block.

Technical

The entity number or ID would be a 32-bit unsigned integer and the same for the entity version.
The entity would be a compound number with a number in its 32-bit less significant and the version in its 32 bits most significant.

This is an implementation proposal example:

Remove entity operation: it's given when the entity is from at least one component to zero. This increment the version component and add to a free-unused entities list. If the next version is a version overflow (or limit). we do not include to that list.
Add entity operation: we first check if is there any free unused entity, if there, we take one.

Use cases

Extending the conversation to its use cases.

0) Local scene usage

This is the most common use case (at least, until now). When you code your scene and call engine.addEntity(), you are having an ID to reference data. Then, you have all the data related to that entity that builds your object.

1) Sync between scene and renderer

renderer has a special appreciation because it shows your scene representation, it completes the image giving the player interaction. In terms of data, the renderer adds the player input to the scene and consume directly the scene responses. The renderer and the scene partially share the same component state.

2) Prefab data

Prefab data is static data that you can add to your local scene.

// Loading a prefab
const staticScene = PrefabApi.load('scene.bin')

// The challenge:
// . 1) Keeping the entities IDs: avoid collision with existing entities
// . 2) Not-keeping the entities IDs: remap entity reference (parenting, indirection): this is not possible with custom components
// . 3) The prefab is another scene with no link

// With the approach 1):

3) Sync between other CRDT peer

We can reference entities with local entities (like an indirection), or mirror the entities.

General and examples

We can reduce all the concepts to one requirement:

To have the possibility of handling a piece of state from another source. The piece of state are components data with entities that can be linked between themself:
1. Prefab
2. A remote scene (server scene)
3. Another scene like avatar, or a portable experience scene

With all these sources, we need to have the capability of interacting with each other. For example parent a local entity with a prefab one, or with another scene entity.

A couple of examples of interactions:

Linking an entity with the root of a Prefab

In this snippet the prefab works with its own entities.

// Loading a prefab
const pfEngine: Engine = PrefabApi.load('scene.bin', 'prefab1') // this takes the all scene state and instance an IEngine object with an ID 'prefab1'
const pfEngineEntity = UniqueIDComponent.for(pfEngine).find('root')

const myEntity = engine.addEntity()
ForeignEntity.create(myEntity, {
    source: {
        prefab: {
            id: 'prefab1',
            entityId: pfEngineEntity
        }
    }
})  

// if the prefab is running as another scene:
//   - it has its own entities, renderer should take it as another scene? as child-scene?

This approach has an inexact path, it seems there are many complexity.

Prefab as a copyable state


function entityMapper(prefabEntity: Entity) {
    const myPrefabStartsAt = 300e6
    return (myPrefabStartsAt + (prefabEntity as number)) as Entity 
} 

// Loading a prefab
const prefabData: Uint8Array = PrefabApi.load('scene.bin')
copyPrefab(prefabData, entityMapper, engine)

function copyPrefab(data: Uint8Array, entityMapper, engine) {
    const crdtState = crdt(data).iterator()

    for (const [entityPrefab, componentId, data] of crdt) {
        const entity = entityMapper(entityPrefab)
        const component = engine.getComponent(componentId)

        // Remove all data and increment version
        engine.removeEntity(entity)

        if (component) {
            const value = component.upsertFromBinary(entity, data)
            // protocol components with known-entity-references
            if (componentId === Transform.ComponentID) {
                (value as TransformType).parent = entityMapper(value.parent)
            }
        }
    }
}

// For default prefab (the ones that only use renderer-components)
//  => The loading would iterate each component that references entities and execute the entityMapper

// For custom prefab, with custom components (like a library)
//  => It should expose a function that accepts all the entities that should be checked to map the referenced entities

With this approach, the complexity is only in the loading, with the remapping entities. The entities that already exist will be replaced and incremented in their version. Link a local entity with a prefab one here, is the same as local with local.

Remote server

The local scene is syncing all entities that the remote server wants to sync. They should share the entities' IDs.

const webSocketTransport = webSocketTranpost(
    'wss://server.decentraland.org/scene_22_11',
    signature,
    function filter(entityId, componentId) {
        return entity > 100e3 && entityId < 150e3
    }
)

engine.addTransport(webSocketTranpost)

// now I have the exact copy of entities from 100,000 to 150,000 here and in the server. 
// This means that if the server creates protocol-components (like MeshRenderer, and Transform, etc), we should see them.

The entities are reserved by the local scene on all peers. With our signature, the remote server can handle some rules to decide our write-permissions.

Remote scene

To access other scene entities, we could create an indirection. We mean remote-scene, a scene that is running in the kernel. Both kernel and renderer know about this remote-scene. Like another process in an operative system, we can share memory, but we need to grant access.

I skip the access negotiation and jump directly to the usage.

Avatar scene

// with the previous negotiation, I know the 'avatar-scene' has to share some entities
const avatarSceneToShare = [75, 76, 77, 78]

// with this I can fetch the data
const avatarSceneEngine = SceneApi.createEngine('avatar-scene', avatarSceneToShare, SceneAccess.ReadOnly)
// SceneApi.createEngine return an IEngine with the transport attached to the directly to the scene 

const avatarEntities = avatarSceneToShare.map(foreignEntityId => {
    const entity = engine.addEntity()
    ForeignEntity.create(entity, {
        source:{
            sceneId: 'avatar-scene', // if the scope is only kernel, maybe scene-number?
            entityId: foreignEntityId
        },
        queryData: true,
    })
    return entity
})

function minimapSystem() {
    avatarEntities.forEach(entity => {
        const avatarSceneEntity = ForignEntity.get(entity).source.entityid
        const transform = Transform.for(avatarSceneEngine).get(avatarSceneEntity)
        const avatar = Avatar.for(avatarSceneEngine).get(avatarSceneEntity)

        // transform.position and Avatar
        updateUserPosition(Avatar.userId, transformPosition)
    })
}

ForeignEntity is a reference to another scene entity. In the renderer, this means that the entity created is got the exact object resource of another scene.

Portable Experience Scene with the current scene

// with the previous negotiation, I know the '0,0' has to share some entities
const entitiesShared = [0, 512, 513]

const sceneId = User.getCurrentScene()

// with this I can fetch the data
const currentScene = SceneApi.createEngine(sceneId, entitiesShared, {
    writeAccess: [512, 513]
})

// While we press IA_PRIMARY we attract all entities into us
function accioSystem(dt: s) {
    const playerPosition = getPlayerPosition()
    if (InputSystem.isPressing(InputAction.IA_PRIMARY)) {
        const Transform = Transform.for(currentScene)   

        for (const [entity, transform] of currentScene.getEntitiesWith  (Transform)) {
            const mutTransform = Transform.getMutable(entity)

            if (Vector3.distance(mutTransform, playerPosition)) {
                const directionVector = Vector3.normalize(
                    Vector3.diff(playerPosition, transform.position)
                )

                mutTransform.position = Vector3.diff(
                    mutTransform.position,
                    Vector3.scale(directionVector, 10 * dt)
                )
            }
        }
    }
}

decentraland / sdk

Entity ID, version and number #500