Polymorph project

[erlend-sh]

https://medium.com/@babylonjs/introducing-polymorph-a55588df6cee

Today, the Babylon team would like to welcome you all to the very earliest stages of our development process. We’ve spotted an industry problem, and we’re thinking about what sort of technology might be used to address it. We’re still very early in the process — we’ve only just begun to write prototype code — so our ideas are far from finished and in some cases are barely even formed. But we’re already very excited by the possibilities, and so rather than waiting to talk about it until after we’ve sanded off the rough edges, we want to open up the conversation to all of you right now. So welcome, everybody! Welcome to the weird and amorphous world of Polymorph!

https://forum.babylonjs.com/t/introducing-polymorph-an-open-discussion/7692/14?u=erlend_sh

Of the many components of Polymorph, the only thing we’ve coded thus far is an implementation of the pipeline abstraction layer: pipeline.h. (It’s actually only one of two planned prototype implementations of the pipeline; for more information about the other, let’s talk on the forum.) This pipeline implementation is built using template metaprogramming to formalize the pipeline as a compile-time concept. While an in-depth explanation of this implementation is well beyond the scope of this blog post (if you’re interested in that, again, please let me know on the forum), there are a few topics that I think are worth discussing here, most of which can be seen in the header’s provided example pipeline.

https://gist.github.com/syntheticmagus/32bff819700363d6ed931471c88f09ce

[AlexEne]

Sounds like gstreamer for mesh manipulation

[aclysma]

Happy to see this @erlend-sh, thanks for adding it!

Relevance to this Workgroup

I think a discussion on asset pipelines would be great for a number of reasons:

• Rust’s combination of speed and safety, especially when paired with cargo, make it a great fit for sharing individual parts of such a pipeline and developing them as a community. In short, this is something Rust should be very good at.
• The asset pipeline is an important part of gamedev workflow, and there are few (any?) offerings that aren’t tightly coupled to a specific engine.
• The asset pipeline is a relatively low-risk place for established studios to introduce Rust, and success there could lead to further usage beyond the asset pipeline.
• A mature, best-in-class asset pipeline could attract smaller teams to use Rust for their entire engine stack

A pipeline implemented in Rust could be genuinely helpful to our community and the broader gamedev community, while also furthering our own desires to see Rust be more widely adopted within this space. I’m not sure that it’s within the scope of this workgroup to take on implementing this, but I think a discussion to explore what we could do to support and foster any efforts towards this could be very beneficial.

Some Other Resources to Look At

There’s a really nice write-up for a pipeline here and there’s probably some overlap with what is proposed in the article that Erland linked. I think the “graph-of-nodes” concept works well and conceptually it’s not far from a material editor in many popular engines. I think it’s a good balance of flexibility, abstraction, and ease of use.

I’ve also noticed that a few of the hyper-scale tech companies are creating their own build systems. (Google’s Bazel and facebook’s Buck) They’re a bit new and aren’t targeted at games, but I think there are ideas we could borrow from them. At a conceptual level they also use a graph structure. They have their own considerations (distributed build farms, build artifact caching) that could be relevant in the games space.

@kabergstrom has been working on atelier. I’ve been getting familiar with it and it looks very promising. The scope of atelier (as I understand) includes:

• Detecting file system changes
• Importing the data
• Building/transforming the data using a node-based system (not implemented yet)
• Putting it into an efficient data store
• Loading the data into the game
  • During development, in a hot-reload-friendly way (notifications for asset changes)
  • At release, in a fast, packed format (not implemented yet)

This may be broader than what was proposed in the article @erlend-sh linked, but that article didn’t address hot-reloading data. They may be assuming a fully “offline” process where the final packaged assets are flat files requested via HTTP. Given that atelier already has to support multiple “output” cases (development vs. shipping), there may be a natural way to extend it to output data in a web-friendly way.

Looking forward to seeing other perspectives on this!