Allow creating custom rendering backends using GDExtension

[ValorZard]

Describe the project you are working on

If I want to work on a game and want to use a custom renderer like https://github.com/EmbarkStudios/kajiya for whatever reason. Or, if I want to have a DirectX backend or some other graphics API, but don't want to wait for Godot 4 to have one in a later version.

Describe the problem or limitation you are having in your project

Right now, if I want to add a custom renderer to replace the native Godot one for whatever reason, it would require diving into the source code and creating a module or changing core Godot code. This is definitely possible, but having to maintain my own Godot fork sounds annoying and unnecessary.

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Using GDExtension, one could extend the current render, and replace it with their own. This would allow for plugins to be developed for custom renderers/porting existing renderers for whatever reason, and it wouldn't require having recompile Godot in order to add it.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

An example in another engine is the rust ECS engine Bevy, Bevy has its own custom renderer, but there is a plugin, https://github.com/Seabass247/bevy-kajiya that lets you use Embark Studio's kajiya renderer instead.

If this enhancement will not be used often, can it be worked around with a few lines of script?

No. This is a pretty big change to the GDExtension system.

Is there a reason why this should be core and not an add-on in the asset library?

This requires changes to the core code.

[Calinou]

I think rendering needs too much low-level access to be doable in a fast enough way with GDExtension. While plugin-based renderers exist in other engines, they usually rely on low-level DLLs rather than something like GDExtension (which is higher-level).

[paddy-exe]

@Calinou Wasn't the point of the new GDExtension system to also allow for custom rendering pipelines? Also, since GDExtension is C/C++ code why wouldn't it be fast enough?

[ValorZard]

adding on to what @paddy-exe said, I thought that the entire point of GDExtension was to make Godot much more of a plugin based engine. In fact, much of the improvements touted for GDExtension has been to allow lower level access than GDNative ever could. Not allowing for changes to the rendering pipeline seem like a strange restriction to have on it.

[mrjustaguy]

Back in july 2021 the idea was to be able to use it for things like physics and rendering, with the two specifically mentioned by @reduz https://www.youtube.com/watch?v=g35xbKWF3ZQ 1:19:43

Strong Emphasis on the idea, as this is nearly a year old, and specifics could have changed quite a bit in that time due to many possible factors... Also it's possible that it is Possible, just not wise due to various possible issues.

Note: The Plugin Physics via GDExtension has materialized, related PR that's been merged https://github.com/godotengine/godot/pull/59140

[Anutrix]

Note: The Plugin Physics via GDExtension has materialized, related PR that's been merged godotengine/godot#59140

Related https://github.com/godotengine/godot/pull/65427 and https://github.com/godotengine/godot/pull/65465.

[clayjohn]

We are definitely working towards being able to create custom rendering implementations in GDExtension. We are in the midst of refactoring now, but one of the main things were are thinking about during refactoring is ensuring that this is possible. That being said, I doubt we will have everything in place for 4.0. It is more likely going to be possible for 4.1

[BastiaanOlij]

I've given this a lot of thought as its something I want to implement in due time. Now there are three versions of it:

1 - Implement a new RenderingDevice backend Basically what Pedro is currently doing with DX12 but in the engine itself. This would allow implementing a different backend that can work with the Forward+ and mobile renderers. So say a metal backend on iOS. In theory this would be a matter of creating a RenderingDeviceExtension class that exposes all the virtual methods of this class to GDExtension, and some sort of mechanism to register it and make it selectable. In theory straight forward, in practice I'm convinced we run into a lot of gotchas and it will take considerable time to perfect this. The problem of doing this now, is that we've been discussing big internal changes to the RenderingDevice API

2 - Implementing a new RenderingMethod implementation This would allow overriding the 3D rendering pipeline in the current RD renderer. Basically Forward+ and Mobile are two different implementations of this, while they both share the same 2D rendering pipeline. I'm on the fence about the usability of this option, while we have tried to keep the RD renderer extendable in practice I've often questioned if this has more con's then pro's and if letting go of this wouldn't allow us to improve the renderer. But if we go down adding this, it's a matter of creating a RenderingMethodExtension implementation and a way to register this and choose this.

3 - Replacing the renderer all together This is basically how the compatibility renderer is implemented.

At it's hart RenderingServer is just a layer that expects a number of core classes are implemented and instantiated, and then routes the calls coming from Godot into the renderer. This involves a number of storage classes that manage the resources on the rendering server (see servers/rendering/storage). This involves a number of environment classes (see servers/rendering/environment) that manage state based on things the user selects in the environment. This involves a class for rendering 2D content (see servers/rendering/renderer_canvas_render.h), one for rendering 3D content (see servers/rendering/render_scene_render.h) and a container class that sets everything up (see servers/rendering/renderer_compositor.h).

If you look at the dummy renderer you also see all these classes implemented, but with placeholder code so nothing actually gets done (well other than Godot doing basic viewport handling and supplying the renderer with objects needed to be rendered. And yes, for those following along because we discussed a GLES2 renderer implementation, that also means that you're free to just implement the 2D bits and ignore the 3D and Godot will be plenty happy that no 3D rendering takes place.

Creating the GDExtension classes to extend all these would be pretty straight forward. The challenge here is to register the new renderer so it can be selected, and how to deal with any needed changes in platforms so the correct context is setup. Right now this is hardcoded to either setting up a GLES3 context, or setting up a RenderingDevice driver, there is no mechanism yet for a 3rd option, such as setting up a GLES2 context or anything else.

To make a long story short, options 1 is a possible way forward for the OP, option 3 is the one for implementing completely separate renderers.

[BastiaanOlij]

Just a small addition, the GLES3 implementation and by extension my option no 3 in its current form, do still use the build in viewport processing logic and culling logic. But adding the option to extend a few more base classes so the RendererCompositor implementation can instantiate those, would allow for the full renderer to be replaced.

[rburing]

@BastiaanOlij

The challenge here is to register the new renderer so it can be selected

I'm not sure how helpful this is, but see

• https://github.com/godotengine/godot/pull/65427

for how this was handled for physics engines (it adds to the drop-down in the project settings).

Edit: Nowadays I think it could be simpler with static methods instead of a whole manager singleton.

[BastiaanOlij]

@rburing thanks! indeed something along those lines would work great. The problem is that when we look at for instance OpenGL, it's not just a callback that does some initialisation. On some platforms bigger changes are needed in how windows are created (for instance on Android we have a different surfaceview).

That said, I think once someone starts implementing this, things will fall into place.

[reduz]

Implementing a custom RenderingDevice IMO is out of the picture, this is waaaaay too hard to make pluggable. For a custom 3D renderer, what should be relatively easy to do is a custom scene renderer, but IMO this is far from prioritary now.

[BastiaanOlij]

@reduz I agree that the custom RenderingDevice is nice from a theoretical POV but probably completely impractical. I just added it to complete the overview.

A custom scene renderer is basically what I sketched in option 2, but I don't think that is suitable for what the OP requested So that leaves option 3

As we've discussed before, totally agree this is not a priority for the core team, we've got far bigger fish to fry at the moment. However if someone else wants to pick this up, at least above is my take on what I think should be the approach to implement it.

[jcarlosrc]

The third option is what I think the original post was intended for. While Godot 3d rendering has come far better still some users would prefer to plug in a completely different renderer to suit their needs. Maybe this could take some priority. As everyone knows one of the main concerns about Godot is its 3d rendering capabilities. This could encourage new users aiming for some specific rendering options to adopt the engine knowing that they are not limited to the in-house rendering server.

[Fz0011]

Is there any chance of the third option being worked on? Rive which has a vector renderer, has shown a lot of interest in this as well! This makes things even more exciting for us. I see many people wanting to use vectors at runtime, and this would be amazing. But for that to happen, this proposal needs to move forward

[Calinou]

Is there any chance of the third option being worked on? Rive which has a vector renderer, has shown a lot of interest in this as well! This makes things even more exciting for us. I see many people wanting to use vectors at runtime, and this would be amazing. But for that to happen, this proposal needs to move forward

I don't think you need to replace the rendering engine entirely to be able to render Rive vectors. In fact, this is probably undesired as you'd have to implement the entire 2D renderer which is a lot of work. This also includes keeping it compatible with existing custom shaders, implementing 2D lighting, etc.

What you probably need to do is to write a custom shader that can render Rive vector data. This is likely what they're already doing for other engines, in fact.

[dragostis]

@Calinou, to get Rive working in Godot we'd need a way to integrate our renderer into the pipeline. We would need access to the GPU context to set up some intermediate primitives (e.g. shaders, buffers, textures) and a texture to render into. A Rive object in Godot can be as simple as an asset with a texture that it renders into.

The third option seems overkill for what we would need. We simply need to override part of the rendering pipeline in order to render Rive assets and keep everything else untouched to it plays nicely with the rest of Godot.

[clayjohn]

@Calinou, to get Rive working in Godot we'd need a way to integrate our renderer into the pipeline. We would need access to the GPU context to set up some intermediate primitives (e.g. shaders, buffers, textures) and a texture to render into. A Rive object in Godot can be as simple as an asset with a texture that it renders into.

@dragostis If that is all you need, you likely have everything you need already exposed (for the Vulkan/D3D12 backends, for OpenGL we need to expose a bit more)

See: https://docs.godotengine.org/en/latest/classes/class_renderingdevice.html#class-renderingdevice-method-texture-get-native-handle and https://docs.godotengine.org/en/latest/classes/class_renderingdevice.html#class-renderingdevice-method-get-driver-resource

[dragostis]

That's great news! I'll start looking into this soon.

[Reuzi3]

That's great news! I'll start looking into this soon.

Is there any news on whether there will be Rive support in Godot? I found a Version but I don't think it's an official Rive release.

[Naming-things-is-hard-btw]

anything new on this?