Construct a render graph from data

[atbentley]

mstr from discords raises some of the difficulties of modifying the default render graph that bevy ships. And some suggestions are raised.

Long term there is a vision to have a render graph editor. Before that can happen there needs to exist some kind of serialized data structure that can be deserialized and constructed into a RenderGraph.

This issue exists to track some of the work that would need to enable constructing a RenderGraph from data.

My current thinking is that as a first pass we could try to replicate the functionality of BaseRenderGraphBuilder with something that gets constructed from a Ron file. See how it feels and decide where to take it from there.

[atbentley]

If no one is already working towards anything like this, then I would begin to do some work here.

Here is an oversimplified api that I would start towards:

#[derive(Reflect)]
struct RenderGraphBuilder {
  nodes: Vec<Node>,
  edges: Vec<Edge>,
}

impl RenderGraphBuilder {
  fn to_render_graph(self) -> RenderGraph
}

[TheLeonsver1]

I hope it's fine I'm saying something that might not be entirely related(if needed please delete the comment). I think it would be very useful if there would be a generic Node Graph editor in the bevy editor. One that can be shipped outside the editor too in user applications. Unlike Unity's Node Graph that as far as I know can only be accessed in the Unity editor and as such cannot be shipped in other applications(you can build custom node graphs inside their editor(like they themselves did with Shader Graph and VFX Graph) but as far as I know, you can't ship it in an application of your own).

[mtsr]

I'm really happy to see this already being worked in, at least in concept. I think this will be a big piece in making things generic enough to be used for games that go beyond the standard pipeline we'll have.

[mtsr]

Here is an oversimplified api that I would start towards:

#[derive(Reflect)]
struct RenderGraphBuilder {
  nodes: Vec<Node>,
  edges: Vec<Edge>,
}

impl RenderGraphBuilder {
  fn to_render_graph(self) -> RenderGraph
}

I think the API as it exists in crates\bevy_render\src\render_graph\graph.rs actually works well enough. The problem is that the different pieces setup in different plugins aren't composable. Currently the BaseRenderGraph is created by bevy_render, with additions being made, specific to the BaseRenderGraph in bevy_pbr, bevy_ui and bevy_sprite.

If you make any significant changes to the BaseRenderGraph, the additions by the other plugins will simply fail to work.

After some reflection, I think that while loading the RenderGraph from config would definitely help, without figuring out a way to make modular additions to it, it would still make plugins dependent on specific RenderGraphs. Or require having a separate config for each of these plugins and combinations.

@cwfitzgerald and I have been discussing this, because it's also relevant to his rendering engine rend3. We were thinking a good first step would be collecting some references. A lot of the material will be pretty in depth on rendering, because in order to make the rendergraph composable, one needs to know the kinds of pieces that will need to be composed.

We've pretty much just started looking. From a quick skim, the top one seems to have the highest abstraction level.

• https://ubm-twvideo01.s3.amazonaws.com/o1/vault/gdc2017/Presentations/ODonnell_Yuriy_FrameGraph.pdf
• http://advances.realtimerendering.com/destiny/gdc_2015/Tatarchuk_GDC_2015__Destiny_Renderer_web.pdf
• http://themaister.net/blog/2017/08/15/render-graphs-and-vulkan-a-deep-dive/

[AlphaModder]

As an interested observer, I would like to emphasize the value of the Destiny GDC presentation @mtsr linked above, especially the parts about the 'render feature' system. The other two links are more about using framegraphs to manage resources and scheduling and to perform optimizations automatically. This is important of course, but it sounds like the central issue here is different:

After some reflection, I think that while loading the RenderGraph from config would definitely help, without figuring out a way to make modular additions to it, it would still make plugins dependent on specific RenderGraphs.

If this is the concern, I cannot recommend the Destiny presentation enough, since architecting a renderer that could be modified in a modular way was one of the main problems the Destiny devs were trying to solve. A big part of their presentation is dedicated to the abstractions they use to make it as painless as possible for their graphics engineers to add new features to the engine. The main advantages of their approach are that:

• It uses the concept of 'render features' to organize rendering code. This is a somewhat higher-level abstraction than a render graph that offers a different kind of composability, where features can be written and controlled independently of each other even when they operate in the same stages of the pipeline. It is well-suited to the implementation of rendering features that involve cross-cutting concerns, i.e. require more than inserting a single 'module' somewhere in the graph (à la Frostbite).
• It keeps the coupling between render feature code and the actual scheduling of render passes on the GPU loose, so that the underlying rendering architecture has some room to change without breaking everything. Obviously there are limits here, a feature written for a forward renderer isn't going to run directly on a deferred renderer with a totally different pass structure, but it makes it much easier to add extra paths to the pipeline or make adjustments across platforms without having to touch all the feature code.

Personally, I think this sort of extensibility would be a big asset for Bevy, both because of the flexibility demanded of a general-purpose game engine and because it would make it easier for independent groups of developers to push the graphics features they want without stepping on each others' code all the time.

[mtsr]

Reading https://github.com/aclysma/rafx/blob/master/docs/framework/framework_architecture.md it seems Rafx also uses this concept of render features. I'm going to dig in a bit to get a better idea of how it works.