Editor refactor: Where to store per-asset data that is editor only?

MechWarrior99 commented 1 year ago

I am working on refactoring the editor, and ran in to the question of where to store per-asset data that is only used in the editor.

Currently classes like the Entity have a field of a EntityDesign class which stores the editor only data. It is then serialized with the asset itself.

There are several issues with this:

It mixes editor data with runtime data.
If there are additional classes that the editor needs to store data in for an asset, those would also need to be included in the runtime section (again, mixing editor and runtime bounds)
It bloats the runtime data.

There are a few pros too:

Easy to access in the editor, especially for things with more complex structures like Entities where each entity needs its own data.
All in a simple easy single file.

Options

1. Editordata files

Introduce a .editordata file that is saved along side the asset files much like Unity's .metadata files (but whose purpose would be rather different). These files would not be visible in the asset browser.

Pros:

Clear separation between runtime and editor data.
Maintains separation between runtime and editor files.
More extensible. Cons:
A little harder to get data for complex structures like Entities where you want the data per entity.
Adds additional files to clutter and need to be messed with, especially when moving files.
Harder to managed, keeping them in sync and users could manually move, rename, or delete them or the asset file without editing the other.

2. Mirrored editordata files

A slight alternative to 1, instead of saving the .editordata files with the asset files, save them in a new fold that mirrors the assets folder. So instead of Assets/Prefabs/Entities/MyCube.editordata, you would have EditorDataAssets/Prefabs/Entities/MyCube.editordata

Pros:

Same as 1 Cons:
Same as 1
More complex to mirror
harder to manually share files.

3. Same file saving

I haven't looked too much in to this one, so it might not be very feasible but still worth mentioning I think. The editor data and the runtime data are both serialized as separate blocks in the same file.

Pros:

Keeps everything in a simple and easy single file.
Possibly easy to manage the data. Cons:
Blurs the line a little bit between editor and runtime data.
Bloats the runtime file with editor only data.
Adds complexity to the serialize.

4. Store in field.

Basically just keep it as it is and store the editor data in the runtime class. Same pros and cons as already listed.

Conclusion

Those are all the ideas I could come up with. There are pros and cons to all of them, please let me know your thoughts and if you have other ideas.

manio143 commented 1 year ago

I don't think you're correct in what you described here - the Entity class present in the runtime has zero knowledge of EntityDesign from the Assets package. And that's the whole idea - libraries in Stride's source code are classified either as engine or as asset (you can see that based on types registered in generated serializers or in the Module.cs). The asset files you see in your project directory are already representing the asset view - i.e. editor time properties. Only when it goes through AssetCompiler the editor time data is transformed into runtime data which would have only required properties.

Maybe I missed something in your question, but my understanding is that adding additional metadata files would only have a purpose if you need to enrich an otherwise sealed class - which the current system of having separate asset and engine types solves already.

MechWarrior99 commented 1 year ago

Sorry, it took a while before I could get back to take a look at the code.

I don't think you're correct in what you described here - the Entity class present in the runtime has zero knowledge of EntityDesign from the Assets package

libraries in Stride's source code are classified either as engine or as asset

While that may be the case, the EntityDesign.cs file is in the engine folder, but the class is in the Stride.Assets.Entities namespace. Now, I definitely haven't read all of the source code, far from it. I have mostly just been reading stuff related to the UI and editor. So the way I am reading it, is that while the EntityDesign may not be used, it is still in the 'runtime' codebase instead of the 'editor'. Which seems to be mixing runtime and design time responsibility. Please correct me if I am wrong or misunderstanding this!

manio143 commented 1 year ago

It's in a different project, though - Stride.Assets. That project is in fact in the engine folder, but that's just physical organization - it can be moved anywhere else. The point of it though is that the classes in Stride.Assets are specifically related to the code in other runtime assemblies of the engine, as opposed to the abstract asset pipeline code in the assets folder. That project would not be referenced by a user in their game code and as such I don't see a reason why it's an issue - maybe that's the point of misunderstanding - the idea that all projects in engine folder are runtime specific as opposed to runtime related.

MechWarrior99 commented 1 year ago

The point of it though is that the classes in Stride.Assets are specifically related to the code in other runtime assemblies of the engine

That is what I mean, it is related to runtime assemblies. Lets say we have a class that holds some editor specific data, but that information is material asset specific. So we want to have that information stored with the material asset in some way. The issue is that the data we want to store is a editor specific class and so cannot/should not be referenced by a runtime class.

Now, I may still be misunderstanding the way that Stride.Assets works/is used. And if that is the case then please correct me.

manio143 commented 1 year ago

Let me draw this out for you:

graph LR;
  UserGame-.->Stride.Engine;
  Stride.Assets-->Stride.Engine;
  Stride.Assets-->Stride.Core.Assets;
  Stride.Core.Assets.CompilerApp-->Stride.Core.Assets;
  Stride.Core.Assets.CompilerApp-.->Stride.Assets;
  Stride.Assets-.->AssetFile(Asset file model);
  Stride.Engine-.->RuntimeModel(Runtime model);

classDef green fill:#b0f449
class AssetFile,RuntimeModel green

The Stride.Assets implements the abstract interface of Stride.Core.Assets and is referenced by the asset compiler (in the future when plugins are implemented it wouldn't be). It defines design time models that during asset compilation phase get transformed into runtime models defined in engine classes.

For example ModelAsset describes a model by filename of the 3d model file. During compilation that data is loaded and transformed into a list of vertices understood by Stride's rendering system and stored so that it can be loaded as the Model class. Model knows nothing about external files, FBX, Blender, etc - it only knows about the Stride's runtime model format.

Why is Stride.Assets related to engine projects? How else can you define an asset class that would be compiled to a runtime class? At the very least the asset compiler class (e.g. GraphicsCompositorAssetCompiler) needs to reference the runtime class to be able to create an object of it and serialize it so that later the game can load it at runtime.

Currently all engine assets are in a single project. Once plugin infrastructure is created we could see Stride.Physics.Design, Stride.Navigation.Design, Stride.Particles.Design, etc. one project with asset definition per one project with runtime definitions. And when that happens we may create a new folder for those projects.

My point is - the separation you're talking about is already achieved.

stride3d / stride