Rework core data model implementation

[Frooxius]

Is your feature request related to a problem? Please describe.

FrooxEngine's data model is one of the most powerful features, upon which nearly everything else is built. It provides implicit synchronization and persistence, with very little effort, giving Resonite it's highly dynamic and collaborative nature.

However currently this comes with certain costs, as some common usage patterns end up causing significant amount of overhead. Notably any behaviors that are continuously changing the data model (such as IK or Dynamic Bones) have their changes explicitly tracked every single frame.

The implementation of the data model has also been done in a way that makes it easier to reason about the data model function, but has significant performance and memory overheads and poor cache locality. There is significant overhead associated with every field on a component for example, both during initialization and runtime.

Lastly, the data model still has some limitations and specifics for particular type of application - a 3D game engine application. Its versatility and use cases go beyond that, but those are harder to leverage with current implementation.

Describe the solution you'd like

I've been working on a design for a new fresh implementation that will resolve the above problems and provide a foundation for many subsequent engine optimizations and expansions.

There are many things that come to this design to describe in detail here, but here is a quick summary:

• Data model will natively support "continuous" drives. These will allow to get rid of overhead for behaviors that are known to change continuously -- This also allows for better multithreading support, as the current data model also requires the changes to be done from a single thread due to the built-in tracking
• Synchronization metadata will be separated out from actual data and fields will be struct based, with IL weaving used to track and flag changes more efficiently.
• RefID allocations will be structured to be done in blocks, with "block owners" - this will handle scenarios like components more efficiently, since those are known to have a number of fields, that are always known and fixed
• Instead of the system structured specifically for 3D worlds, the system will be more generalized and dynamic. The specific behaviors will be provided through "Systems" that are loaded into a "Domains", with 3D worlds being one of them -- This will allow for creating not only 3D worlds, but also 2D, UI applications and other types of applications that have implicit synchronization -- It will also provide a way to create large worlds, by having world split into multiple domains, with each representing a single chunk of the world and other domains coordinating them -- Mixing different systems within a domain will be supported too. This will allow part of the data model to be synced to particular devices, without the overhead of the entire 3D world - allowing for light clients to join (e.g. allowing users to join on mobile devices with mostly just voice data and simplifies representation of the user or using the data model to sync state to simpler IoT devices (like Raspberry Pi) without needing to load the entire 3D world)
• Supported datatypes are communicated/negotiated as part of the session protocol -- This will provide better security, preventing crashes and issues by introducing unsupported types -- This makes it easier to support mods, plugins and other extensions as well and have those co-exist - since the supported types are communicated, the clients will not allow introducing types that weren't agreed on as part of the session - but they can still exist as part of the client

Describe alternatives you've considered

There are no viable alternatives if we want to resolve above problems. Incremental updates to the data model would not be sufficient.

Additional Context

This will not be worked on immediately as of this issue. However this is the first step we'll take when we start working on major optimizations, as the data model is major source of performance issues and a pre-requisite for many more performance optimizations.

Many additional optimizations and applications will be built on top, but this is going to be the first step.

Reworking the data model will likely be a large process too, especially to ensure compatibility with previous one.

[shadowpanther]

• Mixing different systems within a domain will be supported too. This will allow part of the data model to be synced to particular devices, without the overhead of the entire 3D world - allowing for light clients to join (e.g. allowing users to join on mobile devices with mostly just voice data and simplifies representation of the user or using the data model to sync state to simpler IoT devices (like Raspberry Pi) without needing to load the entire 3D world)

Does that mean that for example the Headless would be able to just take the role of syncing and streaming "hub" and discard all 3D info and related calculations, lowering the load on itself?

[Frooxius]

@shadowpanther No, it doesn't mean that.

If you want to host 3D session, it needs to host the 3D systems. You can't really host those without "hosting" them.

We can have the headless skip certain things and calculations that are known to not be necessary, but problem is determining what's not necessary - say someone has behavior that depends on the user's IK being solved and it runs on host (headless) - not solving the IK would break this behavior.

Similarly if there are systems, say culling that determine on the headless who gets what data streams, they still need to be computed.

What you're thinking is more of a "relay", but you'd still need an actual "host" of the session that computes the session stuff.

[shadowpanther]

Okay, that's fair.

[gameboycjp]

I wonder if giving Domain Chunks the ability to be saved separately, or even hosted by external clients, would be viable? Could be useful for, say, a world with a bunch of user-owned rooms.

[Frooxius]

@gameboycjp That'll be inherent property of domain chunks and the major reason for them - they allow for sharding - splitting large sessions into multiple hosting nodes.

[TisFoolish]

Will this allow for stuff like 2D slots that only have an x and y, simple rotation, etc? Will it affect UIX? Having 2D slots could make UIX conceptually easier if their values directly tie into UIX.

[ShadowJonathan]

I saw someone mention the idea of making a 4D space, a-la 4D toys and 4D golf.

Would such a thing be directly possible with domains? Or would it require mods to introduce new datatypes?

[Frooxius]

I saw someone mention the idea of making a 4D space, a-la 4D toys and 4D golf.

Would such a thing be directly possible with domains? Or would it require mods to introduce new datatypes?

Nothing ever requires mods, anything can be implemented if we prioritize it.

The question here is, will we introduce official support for 4D worlds.

I'd say that's more unlikely to happen anytime soon, given how niche that is and how complicated that actually is - it's not just coordinates, but also rendering, physics, interactions and so on. There is a lot of complex stuff you need to do for 4D worlds and most of it you can't get off the shelf.

This means it's likely months, possibly years of work to have a fully featured support for 4D worlds. And given how niche that is, we're very unlikely to make that kind of time investment over other features.

It would have to be more likely implemented through community support. The in-game mechanisms should be sufficient to achieve that.

[BaxterOttoman]

I mentioned an idea at Rucio and he suggested I put it in here. I think it may be useful to have a data type for values that are streamed out which don't need to be reliably received. They can be lost and don't need to be acknowledged and don't cause delta messages.

[Nytra]

That's basically a ValueStream which is already used for some stuff like voice and user position streams, which is why those things never desync. They are just not accessible to use directly in creations right now.

[BaxterOttoman]

Yeah, I think that kind of thing would be useful to have access to for a lot of things.