matrix-org / thirdroom

Open, decentralised, immersive worlds built on Matrix
https://thirdroom.io
Apache License 2.0
610 stars 65 forks source link

Manifold Editor #62

Open robertlong opened 2 years ago

robertlong commented 2 years ago

The Manifold Editor is our game engine environment / asset editor built around the glTF standard. Both the interchange and authoring format will utilize glTF and should be capable of producing content for Third Room as well as exporting glTF content for use elsewhere.

The todo list below represents the tasks needed for a feature-complete 1.0 release. We plan on tackling this list in chunks, focusing on iteratively improving our demo environments.

robertlong commented 2 years ago

Hierarchy Panel and Properties Panel Design:

Editor state is shared from GameWorker to MainThread in two SABs

The first SAB is used to render the hierarchy panel and the second is for the properties panel.

Strings and other non-serializable fields are sent via post message as property changed events.

Every game tick we iterate over all the editor exposed properties of the active component and check for changed properties and copy to the SAB. If the field is a complex value (string, etc) we check for equality and then send a property updated post message. Dirty flags are reset at the beginning of each tick. If the active entity changes we create a new SAB and post message to the main thread. Getters for component properties are defined in component schemas. Complex equality operations should be defined here as well.

For the hierarchy data we set that dirty flag inside our transform component methods on the game thread. All of this data is stored in a SAB and syncs like any other triple buffer. No post message required.

For node names we'll want to create a component for storing names of nodes that will also store a dirty flag. We'll post message any changes to names from the game thread to main thread.

The main thread hierarchy view iterates over all the hierarchy info every animation frame and mutates its representation of the scene graph. It can be efficient about this by only updating the parts that actually changed.

The main thread property panel uses the component schemas it gets from another post message based API and the active entity buffer to render the component properties. Because it reads from a SAB, the inputs will be super responsive for any component property changes. Even if we're in play mode and observing an entity while it's in motion or animating, etc.

Big design points here are:

All mutations from the Editor UI are made via postMessage to the GameThread. Setters for these properties are defined in component schemas. Creating entities will use prefabs. Prefabs are just functions that create entities with pre-assigned components. We'll display a list of prefabs to create when clicking on create new entity. Creating a new entity is accomplished via a postMessage with the prefab id and any initial properties sent from the main thread to game thread.

hazre commented 1 year ago

Hi I had a question regarding the direction of how the Editor will be integrated.

Mainly I want to know if there has been any thoughts around being able to use the editor while in XR. It would be great to be able to directly create UGC and manipulate the scene without ever having to exit VR for example. This is especially useful in a collaborative setting where mutliple people can work on world or experience on any device they're using.

Some great inspiration for this would be for example NeosVR, where almost everything beside assets are created in-game. here's quick video showcasing how the scene editor for example works: https://youtu.be/dqsjidSmlJk?t=197

robertlong commented 1 year ago

Hey @hazre we're slowly working towards a collaborative XR editor but there are some smaller milestones that we plan to reach beforehand. Our next step is to finish the property panel which will allow you to start inspecting all of the data in your scene by pressing the tilde key. Then we'll add single player editing. Then multiplayer and so on. Editing in XR is on our roadmap though!

hazre commented 1 year ago

Hey @hazre we're slowly working towards a collaborative XR editor but there are some smaller milestones that we plan to reach beforehand. Our next step is to finish the property panel which will allow you to start inspecting all of the data in your scene by pressing the tilde key. Then we'll add single player editing. Then multiplayer and so on. Editing in XR is on our roadmap though!

Oh okay sounds good, I just didn't see it on the issue itself, so I thought it wasn't considered. Glad to hear that's on the roadmap. I do wonder though how that's going to work if the desktop UI is in React; would the XR UI be in WebGL? if so, I feel like maintaining two different UI systems might be troublesome, maybe there should be some thoughts in place on a unified UI that works on all platforms?