search

Redot-Engine / redot-proposals

Redot Improvement Proposals
MIT License
33 stars 7 forks source link

Integrating BGFX as the Rendering Backend for the Redot Engine #19

Open nonunknown777 opened 1 month ago

nonunknown777 commented 1 month ago

Introduction

This proposal suggests integrating BGFX, an open-source, cross-platform graphics library, as the rendering backend for the Redot Engine. Redot. By integrating BGFX, Redot could further enhance its rendering capabilities and cross-platform support.

Advantages of Using BGFX in Redot

1. Cross-Platform Rendering

BGFX offers robust cross-platform rendering support. It works on a wide variety of platforms, including:

For Redot, this means easier support and potentially better performance across all these platforms. Redot already targets many of these platforms, but BGFX could simplify maintenance and provide a more unified experience across devices.

2. Backend Abstraction

BGFX abstracts the graphics API and supports various backends, including:

This would enable Redot to switch between different backends dynamically based on platform capabilities, allowing developers to focus more on game logic without worrying about platform-specific details. For example, developers wouldn’t have to manually switch between Vulkan or OpenGL — BGFX would handle this seamlessly.

3. Performance Optimizations

BGFX is designed with performance in mind, featuring multi-threading support, draw call batching, and fine-tuned memory management. This can significantly improve rendering performance for complex scenes, which could benefit both 2D and 3D games in Redot.

4. Cross-API Shader Language Support

BGFX supports shader cross-compilation using shaderc which enables developers to write shaders once and deploy them across multiple APIs (GLSL, HLSL, SPIR-V, Metal). This reduces the complexity of writing separate shaders for each backend that Redot might target.

5. Well-Maintained and Actively Developed

BGFX is an actively maintained project with a strong community and continuous updates. It’s often used in projects like Unity Tiny Mode and other engines, proving its reliability and effectiveness in professional-grade projects.

integrating BGFX into Redot Engine would make exporting to consoles easier, thanks to BGFX's cross-platform abstraction and support for multiple graphics APIs, including those commonly used on consoles. Here’s how BGFX simplifies the process:

6. Cross-API Abstraction for Consoles

BGFX is designed to abstract away platform-specific graphics APIs, meaning it provides a unified rendering layer over different systems. This is particularly useful for consoles, where each platform typically has its own proprietary graphics API. By using BGFX, you can reduce the need for custom backend development per console.

BGFX abstracts these APIs into a single rendering interface, so developers do not need to write specific code for each platform. This makes it easier to support multiple consoles with minimal platform-specific code.

7. Unified Development Across Consoles and PC

The biggest advantage of using BGFX in Redot when targeting consoles is unified development. You will be able to build your game once and have it work across PC, mobile, and consoles with minimal adjustments. Since BGFX handles the platform-specific differences in APIs, the majority of your game code, rendering logic, and assets can remain unchanged, dramatically simplifying the porting process.

Disadvantages and Challenges

1. Potential for Increased Complexity in Redot’s Existing Systems

Redot already has a custom rendering pipeline designed to support both OpenGL and Vulkan. Integrating BGFX as an additional layer might introduce some complexity and redundancies, especially if Redot’s core team decides to maintain both their existing pipeline and BGFX simultaneously. Some of the challenges might include:

2. Performance Overhead on Lower-End Systems

While BGFX is designed to optimize rendering, adding an extra abstraction layer could cause slight overhead on very low-end devices or specific cases where Redot’s current pipeline may be more optimized for that scenario.

3. Dependency Management

Introducing BGFX as a core dependency in Redot would require ongoing maintenance to ensure compatibility between the two projects. Any significant changes in BGFX’s API might require Redot’s core developers to adapt, and vice-versa.

Proposed Implementation Plan

  1. Exploratory Integration: Begin with an exploratory phase, where BGFX is added as an optional backend alongside Redot’s current OpenGL/Vulkan pipelines. Developers can toggle between the default and BGFX for testing purposes.

  2. Benchmarking: Perform comprehensive benchmarks across multiple platforms and device types to analyze performance, stability, and compatibility differences between BGFX and Redot’s current rendering solutions.

  3. Community Feedback and Iteration: Open the integration to community feedback, and gather real-world use cases from developers. Iterate based on the performance, ease of use, and any encountered issues.

  4. Long-Term Maintenance Plan: Develop a plan for long-term maintenance, including how BGFX updates would be integrated into Redot’s core, how it will be documented for developers, and how it will be integrated into Redot’s toolset (like the Visual Shader Editor).

Conclusion

Integrating BGFX into Redot could provide substantial cross-platform benefits, enhanced performance, and a more streamlined rendering pipeline. However, it also comes with challenges, particularly in terms of managing complexity and maintaining compatibility with existing systems. By starting with an optional integration and gathering community feedback, Redot can take a measured approach toward adopting BGFX as its rendering backend, ultimately enhancing its flexibility and power.

SkogiB commented 1 month ago

Big proponent of this down the line, as Godot's heavy NIH design is a huge obstacle to improvement

Spartan322 commented 1 month ago

I'm still not entirely sure a number of concerns raised in godotengine/godot#19602 have changed for Redot, (unless we were to add pluggable render pipelines, but that be a lot of work) if there is another renderer that provides similar things without being as all encompassing as bgfx maybe, (like the shader compiler presents an issue for GDShaders) but I think it still presents a lot of technical issues to do this, I would personally love to be proven wrong on that front though, biggest concern is that we can't just break Godot's rendering API and we can neither deprecate it, if bgfx causes that I don't think it can be viable for our cases.

ghost commented 1 month ago

I'm still not entirely sure a number of concerns raised in godotengine#19602 have changed for Redot, (unless we were to add pluggable render pipelines, but that be a lot of work) if there is another renderer that provides similar things without being as all encompassing as bgfx maybe, (like the shader compiler presents an issue for GDShaders) but I think it still presents a lot of technical issues to do this, I would personally love to be proven wrong on that front though, biggest concern is that we can't just break Godot's rendering API and we can neither deprecate it, if bgfx causes that I don't think it can be viable for our cases.

In the long run though, even if it is API breaking, it might still be the best approach. As it would alleviate a lot of the workload now resting on a relatively limited number of shoulders. Not even addressing the issue of BGFX being a very mature rendering engine. Effectively far more mature than any of the renderers present in Godot/Redot. And therefore more reliable, less bug prone. Which could on its own already resolve a lot of issues that may or may not exist with the currently present renderers in Godot/Redot.

Yes, it would be a lot of work to implement BGFX but I would say it is at the very least worth investigating as it could end up proving incredibly valuable down the line. OK, it would probably not something that could be done overnight nor should it be of a very high priority but, creating a branch for it and simply toying with it to see how viable or not it is is worth it, I think.

Edit: Regarding the issues raised in the issue you linked: There's this

Bit of a reductive argument against it.

Spartan322 commented 1 month ago

I would be massively opposed to anything that does that to the API, and I don't see that it would inherently be more reliable, less bug prone, and/or just as if not more performant, that requires proof in at minimum some rudimentary demo inside of Redot to justify consideration anyway.

Edit: Regarding the issues raised in the issue you linked: https://github.com/godotengine/godot/issues/19602#issuecomment-435049950

Also referencing this is unfortunately a copout, yes its a terrible and dumb response, however previous concerns stated by the Godot maintainers are not wrong, pointing to a bad response from someone who doesn't want to argue anymore shouldn't be convincing to either side.

Personally I think the bigger issue is that if someone really finds this a concerning problem to resolve, the minimum we need to see if its even viable is a demo, it doesn't need to be feature complete, it doesn't need much, but somebody somewhere has to make something that at least demonstrates its worth investigating, and I don't think the onus on doing that should be the Redot team but someone who actually thinks this is a good idea. There's enough work to do already and any upcoming roadmap should be made of concrete solutions, (Godot was full of enough vague promises, I would desire clear milestones and goals to complete) without a demo this unfortunately sounds like a pipedream, even if one I do wish we could have because I do agree having a dedicated battle-tested solution would be better then using the ad-hoc solutions done for Godot.

Starkium commented 1 month ago

what is unreal engine doing? From all the things I've played with, it has produced the most performant low-end output (assuming apples to apples comparison of scenes)

SkogiB commented 1 month ago

what is unreal engine doing? From all the things I've played with, it has produced the most performant low-end output (assuming apples to apples comparison of scenes)

In house it seems, from what I could find in web searches

nonunknown777 commented 1 month ago

I would like to ensure that in the Godot Engine at its current state all rendering stuff is done/maintained by few people, mainly reduz. By using BGFX we could have a very mature and decent rendering system (which will be maintained by BGFX devs, same happens by using jolt physics, also we could help to find bugs if any). The other option is always keep in sync with godot which I think will make redot not very different from godot, this is more relevent than you think...

ps: You guys also knows that godot always change their rendering system and we never had a fully working system (not trying to undo what reduz did which was a gr8 job, but you know we had if I remember well like 2,3 different rendedring systems). The hard work will be implementing it, but the effort to keep it updated will be way less than current godot engine way it does currently!

smy20011 commented 1 month ago

Multi-threading: BGFX utilizes multiple threads for rendering, reducing CPU bottlenecks.

This one smells like ChatGPT. Almost all engine use GPU to render and I don't think CPU is the bottleneck most of the time unless you have 10k+ objects to add and remove every time.

SkogiB commented 1 month ago

I would like to ensure that in the Godot Engine at its current state all rendering stuff is done/maintained by few people, mainly reduz. By using BGFX we could have a very mature and decent rendering system (which will be maintained by BGFX devs, same happens by using jolt physics, also we could help to find bugs if any). The other option is always keep in sync with godot which I think will make redot not very different from godot, this is more relevent than you think...

ps: You guys also knows that godot always change their rendering system and we never had a fully working system (not trying to undo what reduz did which was a gr8 job, but you know we had if I remember well like 2,3 different rendedring systems). The hard work will be implementing it, but the effort to keep it updated will be way less than current godot engine way it does currently!

A good solution for large updates like this would be Addons. Dragos and others in the engine chat discussed these, it's definitely on our radar as a way to overhaul the core systems without breaking compatability

meganuke commented 1 month ago

Hello everyone!

A long time ago, i checked Godot, it was version 2 iirc. since im an enthusiast of graphics engines, i went straight to the source code to check the rendering system....

and couldnt find it...

so i posted in the forum and somebody (that same reduz guy iirc, but may be not...) directed me to the code.. it was (and still is) in a folder called "Drivers".... so that was my first impression "ok, as talented as they could be, the people working on this dont know what a driver or a renderer is". my second impression was bad too, as there were absolutely no coding standards. (they do have some now).

so then i asked "why did you decided to create your own renderer instead of using a mature one like Ogre3d?" and his answer was "what do you mean by mature?". so third impression: "these people dont even know what mature software is..."

after that i stayed away from godot, and to anyone how asked me i told them to do the same.

i believe if you have to choose between talent and experience, choose experience, its waaaay more valuable.

all that to expose my point: have you checked the code of bgfx? it has all the vulkan rendering code in 1 file, 9482 lines of code.... dont believe me, go and check it out! 9482 friggin lines in 1 file....

so, if you really think you should consider a new renderer, consider Ogre-next. https://github.com/OGRECave/ogre-next but keep in mind a few things: 1) it will take time 2) it will be painful 3) it wont be easy. 4) it will take time... a lot

my suggestion if anyone is truly interested is to start working on it without making it public until you have a working prototype.

Spartan322 commented 1 month ago

Please keep issues on topic, if you have a suggestion for another rendering backend, that should be proposed in its own issue.

mohsenph69 commented 1 month ago

I think Godot renderer is good enough it is better to put effort on making better Godot rendering server! I think in Redot one of the most important priority should be adding a good and reliable global illumination system! SDFGI system is not good! I don't know about hddagi!!

Spartan322 commented 1 month ago

I can't see anywhere that Filament supports DirectX, (as its intended to be an embedded, as in mobile, rendering engine) such a proposal should be in its own separate issue.

MachinistJohn commented 1 month ago

my bad i suppose it does need to be in its own issue i'll delete it. Why does it matter if it doesn't support directx it supports vulkan?

Spartan322 commented 1 month ago

DirectX support is necessary in order to seek Xbox console support, lacking it makes it inferior to Godot's renderer, and I don't see how an embedded rendering engine is all that useful to a desktop game engine.