Add support for Declarative UI

[Muchaszewski]

Describe the project you are working on

Any UI-rich game - think all of those mobile "Excel" games. They do not necessarily have a lot of gameplay but a lot of UI. This would also be helpful for all simpler games.

Describe the problem or limitation you are having in your project

Editor UI in WYSIWYG mode is great if you are doing simple UI, and do not plan to support anything else than one aspect ratio. With the rising requirement of writing for more and more resolutions, windowed modes, different aspect ratios, and so on, it's important that when doing UI we would be equipped with the best tools possible!

That being said, coming from the world of Game Development and doing enterprise for a few years now, with web frontend I must admit that the way web browsers deal with UI is superior compared to WYSIWYG editors.

It's trivial in React.JS, Angular, or a plethora of other web UI frameworks to do fully reactive UI.

Given a simple comparison. We had a case at my company where we had to port UI from Unity to React.JS, and given the exact design, API, and end result. We ended up writing the UI in Unity for 5 months, and we finished the same work in 3 months in React.JS. The comparison might not be totally fair, as this was done by different people at different times, but my point remains.

To list some issues:

• Cluttered hierarchy "Scene" view with thousands of nested objects
• Requirement to name everything or you will quickly lose track of what is what
• Reactive UI almost always requires custom scripts for - breakpoints, edge cases, custom layouts, special cases

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Given that we could leverage UI as code, and Style as Code a bunch of tools would be suddenly available to help develop games quickly.

• Copy styles from design software like Figma/Adobe XD or plenty others
• Import whole components from design software like Figma/Adobe XD or plenty others
• Quickly make changes in "developer friendly" way to all components or single components by smart code
• Easily nest, expand, and program components via code rather then convoluted UI

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

The goal would be to have an HTML/JavaScript interpreter (or any other language but IMHO it's best to stick with something there is industry around) that would allow for rendering UI in Godot as currently in the web browser. And allow for importing currently supported frontend frameworks like React.JS, Angular, or many others. For CSS a framework like Tailwind

As a smaller goal a new "inspired" syntax for code declarative UI. C# example

Component DoRender()
{
    var button = new Button(label: "This is a button", style: "mb-3 text-bold font-size-16");
    var image = new Image(src:"path/to/image.png");
    button.Add(image);
    return button;
}

If this enhancement will not be used often, can it be worked around with a few lines of script?

For mobile, we could expand the native part of the UI, for Android there is a reasonable declarative UI framework, that could be used. There could be also an embedded React Native with Godot.

For desktop, I in the past through about Electron engine and render inside of a canvas, but this would not work for games, as the delay in Electron to Canvas/iFrame is just to great, and would require huge development effort.

For web, it's solved obviously.

Is there a reason why this should be core and not an add-on in the asset library?

If this would be a library or an addon I would not mind, but having something like this as part of Godot would be a great efficiency boost for all developers.

[Calinou]

Editor UI in WYSIWYG mode is great if you are doing simple UI, and do not plan to support anything else than one aspect ratio. With the rising requirement of writing for more and more resolutions, windowed modes, different aspect ratios, and so on, it's important that when doing UI we would be equipped with the best tools possible!

With the way game UIs are designed, you generally don't need to have responsive breakpoints like you would on a website. Instead, use anchors and the canvas_items stretch mode. This automatically scales your UI to various resolutions and aspect ratios like they'd do in most modern games (AAA games do it this way). See Multiple resolutions in the documentation.

The goal would be to have an HTML/JavaScript interpreter (or any other language but IMHO it's best to stick with something there is industry around) that would allow for rendering UI in Godot as currently in the web browser.

This would require integrating huge libraries in core Godot, which is a no-go for binary size (and security) reasons. See https://github.com/godotengine/godot-proposals/issues/960.

[Etruscian]

So I guess anything that looks like Unity's new UIToolkit is out? That's a shame since it actually works pretty sweet, being able to put all your styles for a UI control in a single file and do things like positioning and even state change styles like hover etc directly from there.

But I do see the enormous effort required to get a system like that stable.

[YuriSizov]

and do things like positioning and even state change styles like hover etc directly from there.

That would go against Godot's node system. You can still style your elements using one Theme resource, but individual components are controlled by nodes and information about their size and position belongs to the node. Besides, parent nodes affect how children are positioned and sized, specifically Containers. Having some detached definitions would be misleading.

In short, Godot has its own GUI system and we don't see a reason to rework it, because it works great and is very versatile. Bugs need to be fixed, but there is no justification for an overhaul.

That said, an alternative GUI system can always be introduced as a module, an extension, or even a GDScript plugin.

[GrammAcc]

I am also a web developer though not as experienced (also primarily backend), and I have often thought that being able to style Godot's UI with CSS would be great, but I would be very hesitant to use modern JS libraries in a game. Frameworks like Angular and React bring in a huge number of dependencies that are impossible to audit, and they are also not built to the performance standards of real time games. There are advancements being made in high performance 3d rendering on the web, but it's nothing like modern desktop game dev. Also, the state model is very different. In React and other SPA frameworks, you rerender the page or modify the DOM in place whenever the server reports a state change to the FE, but in desktop applications, you are running an event loop that redraws sections of pixels on the screen in response to certain machine instructions. Most game applications are closer to implementing the browser itself than implementing the pages the browser is displaying, so outside of the HTML5 export target, I'm not sure how this would really work without a lot of memory and CPU overhead since you would essentially have to run the native UI that Godot is already running and embed the JS UI inside it.

Another potential issue with using something like CSS (though I do love tailwind) for game UI is that CSS and HTML are markup, that is, they are intended to describe a document, not a user interface. The web has evolved into a very interesting interactive playground, but I think that constraining the flow-based nature of CSS and HTML to ensure a consistent and immersive gaming experience would be really hard. A short FOUC on the web isn't a big deal, but it could get you one star reviews on Steam, so any implementation would likely have to prerender the CSS, which makes the implementation much more complicated than simply embedding V8, and prerendering would remove the dynamic flow that makes CSS so powerful, so even if we can figure out how to implement it, we still have the same issues of how to deal with layout on different resolutions and form factors. I guess maybe we could prerender at first run like we do with shader caching, so we have access to the running machine? That might not work if the user switched monitors though, so edge cases would need to be considered as well.

One other thing to consider is that most game devs are familiar with C++ or C#, and it might just be my echo chamber, but I don't really hear about JS being used in game dev much, so I'm not sure if this would actually improve developer efficiency. I started with Godot about 6 years ago, and I also did some work on desktop apps in QT with Python before I ever did anything with the web, and FE work in JS and CSS is still hard for me. I actually find Godot's UI much easier to work with than React (though Vue's component structure makes a bit more sense to me coming from OOP). I think a large portion of DX really just comes down to what the programmer is used to working with.

I guess the kinds of games that are described in this proposal are more point and click or menu-based games, so maybe the performance hit isn't a major issue, but if it were me, and I wanted to use React or another JS framework for a game like that, I'd just build the game in a Node project and host it on my website, or use React Native and deploy to the Play Store. For that type of game, I don't think the power I would get from using a game engine would be enough to justify the additional complexity of integrating a JS framework with it, so I would probably just use Node and call it a day. Personally, I would still use Godot for that type of game though. :)

My very long 2 cents.