search

bevyengine / bevy

A refreshingly simple data-driven game engine built in Rust
https://bevyengine.org
Apache License 2.0
36.41k stars 3.59k forks source link

Editor-Ready UI #254

Open cart opened 4 years ago

cart commented 4 years ago

This is a Focus Area tracking issue

Before we can start work on the Bevy Editor, we need a solid UI implementation. Bevy UI already has nice "flexbox" layout, and we already have a first stab at buttons and interaction events. But Bevy UI still needs a lot more experimentation if we're going to find the "right" patterns and paradigms. Editor-Ready UI has the following requirements:

Active Crates / Repos

No active crates or repos. Feel free to make one. Link to it in this issue and I'll add it here!

Sub Issues

No active issues discussing subtopics for this focus area. If you would like to discuss a particular topic, look for a pre-existing issue in this repo. If you can't find one, feel free to make one! Link to it in this issue and I'll add it to the index.

Documents

Kleptine commented 4 years ago

I wanted to share a few quick thoughts on UI systems in games. I'm a gamedev professionally, and I read this list of requirements, and had a bit of a knee-jerk reaction. These are good features from a basic point of view, but sort of miss the boat on what I would consider are the harder questions around UI:

I'm happy to elaborate on any of these, but primarily I'd recommend trying to learn from where Unity is at the moment. They started with an immediate mode UI engine for both the Editor and the Runtime. Later, because of performance and flexibility, they adopted a retained mode UI for the Runtime. Now, as of two years ago, they've finally been building a (really fantastic) unified UI system for both the Editor and Runtime, open source.

This is a great talk on the new design (UIElements/UIToolkit): https://www.youtube.com/watch?v=zeCdVmfGUN0

I'd highly recommend taking as much learning as you can from their path. Frankly, both previous systems were a total mess. They seemed great in the moment, but just collapsed under any sort of complexity.

What makes UI challenging is that it's very hard to build incrementally, you have to plan a lot of these things in from the start. I don't think Bevy UI should be nearly as complicated as the latest Unity UI, but it's really worth learning, and not repeating their mistakes.

Kleptine commented 4 years ago

And also, if the intention for Bevy is to start off as a hobbyist engine, incremental is perfectly fine. I would just expect to have to rewrite the UI a few times as you want to attract larger projects.

thefuntastic commented 4 years ago

Some early distillation of thoughts is happening here: https://hackmd.io/lclwOxi3TmCTi8n5WRJWvw?view

At the moment anyone is free to add their particular domain experience. This is stopgap - suggestions are that Cart takes ownership of the hackmd / it's used as ingest for a github wiki/markdown page

tektrip-biggles commented 4 years ago

Would it be worth also looking as SwiftUI as an example of prior art?

erlend-sh commented 4 years ago

Would it be worth also looking as SwiftUI as an example of prior art?

Add your write-up of it to the doc if you’d like 👍

stefee commented 4 years ago

Can we add "screen-reader friendly" and "multiple input types" (keyboard, mouse) as hard requirements for any UI framework / UI primitives?

cart commented 4 years ago

At the risk of being very ignorant, is screen-reader friendliness a high priority for a visual editor? I have a feeling visually impaired folks will have difficulty getting use out of the editor. Making a visual scene editor friendly to visually impaired users sounds like an entire research project. Mouse + keyboard navigation is a clear win for accessibility so I think I'd want to focus efforts there, given our limited resources.

I'm certainly not saying we shouldn't make ourselves screen reader friendly, as that will absolutely be useful in other apps, I'm just questioning if it should be an Editor-UI priority. I'm curious what the use cases would be for screen readers + visual editors.

Please tell me if I'm wrong here.

stefee commented 4 years ago

Yeah no worries, it was a genuine question. I'm not sure how well screen readers are supported in other editors, I am coming from a context of developing on the open web and it is really important in that context. I imagine native software is a lot more complicated.

The extent to which it is important comes down to how visual the editor is. If there are lots of text inputs (e.g. a way to change transform values of an entity numerically rather than just using click + drag) then I think screen reader support would be something to consider. If at least in the short-term we are looking at a more purely graphical interface then it is less important.

stefee commented 4 years ago

Also for the record, I don't use screen readers so I don't actually know what users would prefer in this case. Maybe folks would rather just edit the RON files directly. Let's not worry about it too much. :-)

tektrip-biggles commented 4 years ago

Have been looking at SwiftUI and one of the nice things about the declarative approach is that it does mean your data is well structured for adding things like that in the future.

The general concepts they use are really elegant and I suspect would translate well to Rust, although it may be an "all or nothing" commitment to go that route.

Main principle is defining a tree of very lightweight units composed together to create consistent and predictable results with surprisingly little boilerplate and small number of primitive/leaf components. Data dependencies are all declared up front so the framework knows what to update as & when the underlying data changes and there's an "environment" concept so that data can be passed down the hierarchy without needing to be passed through every hop.

I quite like the bounds calculation process too, where parent gives its children size "suggestions", they respond with their desired size and parent determines ultimate positioning (but has to respect child's desired size)

Worth watching the WWDC '19 & '20 talks on it all for ideas.

regexident commented 4 years ago

@tektrip-biggles: FYI, @raphlinus has done a bunch of great research in this whole topic, as well as in comparison to existing systems like SwiftUI:

https://raphlinus.github.io/rust/druid/2019/10/31/rust-2020.html

Kleptine commented 4 years ago

A few thoughts:

Screen Reading: If the goal is for Bevy UI to support in-game UI as well as the editor, that means the UI will need to support gamepad-only and keyboard-only navigation of UI from the beginning. So in a lot of ways, you're already halfway there to basic screen-reading support. The bigger question is one of UX design (ie. does the Editor require the mouse?), rather than implementation.

FRP: Modern FRP-based approaches work fantastically for the web. I think they're really strong when you have a flat UI that is structured as a hierarchy. Game UI can be much more complex. In-Game UI often isn't a flat plane, nor a tree. There might be a myriad of text cards, health-bars, holographic displays, etc. Depending on the game, it may be hard to treat this as a single tree.

Additionally, there are entire projects currently figuring out FRP in Rust (Yew, etc). It's been a massive undertaking, and most require lots of macros, templates, generics, etc. And that's without having to build a renderer. So I worry about complexity, scope here.

What I'd favor is a general, high performance UI rendering engine, integrated with the ECS. An FRP crate could be built on top of it, but wouldn't be explicitly baked into the solution. That would allow UI-heavy and 2D games to use FRP as needed, but not require jamming all other use cases inside of it.

ShalokShalom commented 4 years ago

I think Godot can serve as a nice project to look at, while developing the BevyUI

ncallaway commented 4 years ago

@Kleptine

It's been a massive undertaking, and most require lots of macros, templates, generics, etc. And that's without having to build a renderer. So I worry about complexity, scope here.

What I'd favor is a general, high performance UI rendering engine, integrated with the ECS. An FRP crate could be built on top of it.

I think that's right. Especially the part about complexity and scope. I really really like developing UI in the FRP style and would love if Bevy supported it, but it's a mountain of work that we don't need to take on right now.

I think really nailing a UI rendering engine and ECS would us a good foundation to build different higher level UI experiments on. I could see a React-like UI system someday that treats the ECS UI the way React treats the DOM.

stefee commented 4 years ago

I 100% agree with this @ncallaway and I've had similar thoughts over the last few days. The high-level API is something that can - perhaps should - be developed only once the underlying system has been established and is found to be performant and fit for purpose.

If it means writing very verbose or repetitive code for the time-being, that is a worthwhile trade-off IMO.

stefee commented 4 years ago

Also agree with @Kleptine that the high-level stuff can be user-land crates for the time being.

stefee commented 4 years ago

One point @Kleptine

  • Styling: CSS-like, or in-code?

Why not start in-code and then we can add some optional CSS-like solution later (I'm thinking something like how CSS-in-JS works, i.e. it mostly compiles to target language at build time so everything is static at runtime).

Kleptine commented 4 years ago

I could see a React-like UI system someday that treats the ECS UI the way React treats the DOM.

Precisely my thoughts as well. The ECS would be a great way to store this information in a flat structure. You should take a look at the way Unity stores their UI components in a raw buffer (in the UIElements design talk). It's fairly similar.

Why not start in-code and then we can add some optional CSS-like solution later.

I think that could work out. Unity used in-code styling for their IMGUI implementation. One of the challenges was just that it was cumbersome to feed through all of the different styles through all parts of your program. It might be better, though, if styles can be stored as constants, somehow and exported statically.

So I think some more succinct form of styling would be nice. A CSS-like alternative could be added as a crate, although it might make the API a little more challenging to design. But I agree it's a fairly big task, probably too big for now. Personally, I would be averse to anything other than a CSS subset. There's enough UI styling languages out there that there's no need to reinvent the wheel.

Edit: Another downside of code-only styling is that you need to recompile to see changes. Bevy is already fast to recompile, but you might still have to spend a few minutes replaying the game to get back to the UI you were looking at. It'd be ideal if styling were an asset that could be hot-reloaded in-place, just like any other asset.

stefee commented 4 years ago

It'd be ideal if styling were an asset that could be hot-reloaded in-place, just like any other asset.

Anyone please correct me if I'm wrong, but I think styling as it currently works in Bevy UI can be saved/loaded from scene assets at runtime.

It might be better, though, if styles can be stored as constants, somehow and exported statically.

I was assuming that this would be the case, or at least that UI styling would be built up out of composable functions (i.e. "mixins"). I think this would already be quite easy to do with Bevy UI, but I haven't actually tried it.

stefee commented 4 years ago

I wrote in Discord about the possibility of adopting the Every Layout (https://every-layout.dev/layouts/) primitives as our "building blocks" for layout. I think we could potentially just copy the CSS from the Every Layout components into Bevy UI as some sort of "style mixins".

I'm a big fan of their composable approach to flexbox layout primitives, and since Bevy UI currently uses the flexbox model anyway this would be a good fit: https://every-layout.dev/rudiments/composition/

P.S you have to pay for access to the book, but the CSS itself is not patented, so we can use it.

ndarilek commented 4 years ago

As to screen reader support in the editor/UI, I can speak a bit to that.

I'm a blind game developer working on adding accessibility to Godot. It may not be as relevant here as it is in Godot since I gather that more code-based workflows are first-class here in a way they aren't with Godot, but a few use cases I have in mind:

I'm running up against some Godot limits that make accessibility challenging to implement, so tentatively pencil me in as willing to help with Bevy UI accessibility. My big condition for helping out is that it be baked directly into the UI (I.e. separate components are fine, but I'd want to ship it as a UI requirement that someone might choose to disable for whatever reason, than as a third-party crate.) I'd also like for it to be as integrated with the repo as is the UI crate, such that CI failures breaking accessibility are blockers. IOW, I'm fine with people not launching the screen reader system if they'd rather not, but I'd want UI consumers to automatically have it and be assured that it works with the most recent UI. Hope that's acceptable.

In terms of making my job easier, here are two bits of advice:

  1. Make keyboard navigation first-class. Godot has this problem right now. The tree widget is absolutely broken for keyboard navigation, and fixing it isn't a priority. I'd do it myself, but since I can't see what I'm doing, it's like coding with a strip of cloth between me and the keyboard. Not saying it needs to be exhaustive and platform-specific, but keyboard/gamepad support should be consistent, and fixing breakage should be prioritized.
  2. Send events for just about everything. Focus enters/leaves a widget. Text is added to, or removed from, a TextEdit. Focus/selection moves around an editor. I need to intercept just about everything and provide a speech representation. Godot has some lacks here too, and there's been a bit of resistence to adding signals I need, meaning accessibility becomes more and more hacked-on.

Anyhow, hope that helps. Sorry for going long. I'm about to launch an accessible Godot game, but am happy to try Bevy for my next title and work on accessibility. One good aspect of audio-only games is that they don't exactly require lots of resources--a blank screen and good audio are usually enough. :)

ncallaway commented 4 years ago

I'd vote for accessibility / screen-reader friendly support as part of Editor Ready UI too. I think @ndarilek lays out great reasons why accessibility is important in the Editor itself.

The other reasons why I'd vote to tackle it as early as possible (even if it does expand scope somewhat) are:

I don't necessarily think we'll be able to get the first version of the editor to be in a place where it integrates with popular screen-readers on all platforms, but I would really like the core of the UI system to at least have all the pieces in place so that if someone wanted to add screen-reader support the UI system doesn't get in the way.

ndarilek commented 4 years ago

To clarify, as much as I don't like this route, I don't believe screen reader support is the way to go. Instead, I'd advocate for the approach I took with godot-accessibility--build a screen reader into the engine. Each platform has its own accessibility API, so screen reader support means supporting that entire wide surface area. Additionally, I'm not sure that natively-rendered UIs can export accessibility information on Android. It's certainly possible to improve the accessibility of custom widgets, though I'm not sure that support extends to arbitrarily-drawn pixels on a canvas. By contrast, the custom screen reader approach just requires platform-specific text-to-speech, for which I've already developed the tts crate which supports all major platforms except for Android.

I don't advocate this approach generally, nor would I oppose eventually putting in the work to bridge to platform-specific accessibility APIs. Most of the challenge in game accessibility seems to involve changes to the game rules and world itself (UI-heavy games not withstanding) and I don't perceive that game UIs are necessarily as expansive as, say, a web browser or office suite. An embedded screen reader might not get us 100% of the way there, but it'd get us close enough for all intents and purposes, and would be much easier to implement.

stefee commented 4 years ago

Thank you so much @ndarilek!! This is really insightful.

Ideally, and especially because we have prior art in the form of Godot, we shouldn’t need to rely on blind/partially sighted contributors to implement this. So sign me up as well.

stefee commented 4 years ago

Copying @ncallaway's comment from Discord.

@ncallaway: From the todomvc I've been working on the missing pieces that'd be useful from the UI system:

stefee commented 4 years ago

Based on discussion on Discord, I think #195 should be a priority right now. I'm keen to get going on some of the styling stuff but I don't want to have to re-do too many bits for DPI scaling.

j16r commented 4 years ago

There is an OpenGL accelerated GUI framework in Makepad: https://github.com/makepad/makepad which looks incredible, though the code doesn't seem super well documented nor easily usable outside of makepad.

follower commented 4 years ago

@cart At the risk of being very ignorant, is screen-reader friendliness a high priority for a visual editor?

Really appreciate @ndarilek took the time to share their own experience with you.

I can also recommend reading the "Why?" section of @ndarileksgodot-accessibility` README. (Also I would like to acknowledge @cart's willingness to admitting not having knowledge around this issue & being open to learning. )

Examples like "Working alongside sighted developers who would prefer a visual editor" really highlight to me the importance that the tools we create not exclude people from being part of a development team by being inaccessible.

And, the reality/virtue of developers being "lazy" means that, as suggested by @ndarilek, the best approach is to have the support included in the base system, enabled by default.

When @cart announced Bevy it was only a few days after Godot had received significant (and justified) criticism in relation to accessibility in this thread on "Making Advanced GUI Applications with Godot" and I'd intended to drop a link to the comments (which I'd recommend reading for more context/perspectives) but hadn't so I'm glad @stefee raised accessibility as a consideration early on.

For people on the Twitters, I can also recommend following \@ianhamilton_ ("Accessibility specialist, helping studios avoid excluding gamers with disabilities") which I've found helpful for gaining further insight into both the positive & negative sides of the current state of accessibility in relation to games.

JBradBarnes commented 3 years ago

Given the abundance of UI libraries and frameworks using Google's Material Design Language Specification, the Visual Components are an already very well know abstraction for front end devs: https://material.io/design

The spec answers a lot of the UI abstraction questions better than anything else I've seen.

tyrylu commented 3 years ago

I'm also a visually impaired developer, so sign me up for any accessibility related testing and help. I as of now have nothing to add, everything was said there before, this comment is basically a I am there as well kind of message.

alice-i-cecile commented 3 years ago

Notes are being assembled on hackmd to track Discord conversation.

alice-i-cecile commented 3 years ago

Interesting prototype from @james.i.h on Discord: https://gist.github.com/jihiggins/bd4e6cfe76a28d8913d641ea2ea9ad65

rope-hmg commented 3 years ago

For styling I think it would be good to take inspiration from the tailwind CSS framework. I personally hate working with CSS and I really hope that Bevy doesn't adopt it or something like it.

faramozzayw commented 3 years ago

I personally hate working with tailwind and love working with CSS and I really hope that Bevy do adopt it or something like it :)

Instead of tailwind i would recommend taking inspiration from bulma, its have utility and component-like classes. I think Bevy needs more built-in components/widgets than it does right now

ShalokShalom commented 3 years ago

As inspiration:

minecrawler commented 3 years ago

My 2 cents: I think choosing the way to go is a clash between major fractions of the frontend community.

There is a big chunk of people, who love CSS, maybe because they do it as their day job and feel comfortable with it. They may want to use bevy as a hobby engine. Some may work on AAA titles, as there are also AAA HTML/CSS UI libraries available - example: Coherent Gameface.

There are people, who despise CSS. That's mostly people who do little to no work with it and feel overwhelmed because they have no clue how to structure and work with it to make everything fall in place. My fear is that this is mostly the AAA folk, who do no-/low-code (as seen in Unreal Engine and Unity, for example) and people who don't use the web (mobile and desktop devs).

This leads me to believe that there are three distinctively different main target audiences: The "web lobby", the "please-no-web lobby" and the "low-to-no-coders".

Bevy wants to be an engine by and for developers - as it reads on the website, so I assume it's for everyone who develops a game, including the three distinctions above. Getting all of that onto one page seems to be near impossible to me, if it all should be core bevy.

Which is why I would suggest to go and spike the issue. What do all of the paths have in common conceptually? What kind of properties do they set and how does layouting work? What happens when multiple screen proportions (4:3 vs 16:9 for example, but also mobile vs tablet and similar) and resolutions (1280 x 720 vs 2560 x 1440 and similar) have to be targeted? It may be a good idea to create a comparison matrix and have it filled out by different developers. Make it viewable to everyone and foster discussions, so that modern best-practices are used. Then come up with a base for them all (leaving out their respective golden kitchen sinks and special features) and set that as the bevy internal standard. From that point, plugins can take over, so that there may be a bevy_web_native_ui, bevy_tailwind_ui, bevy_flutter_ui, bevy_omg_is_my_ui_nice, bevy_whatever_its_ui, and so on can exist and implement whatever standard they want. Bevy users can then explicitly install the one they want and not care for the others. Some of them may be officially endorsed and land examples in the book, so people can get started faster.

For the editor, that would mean either building on the internal standard, or developing one of the plugins in the process. The former sounds cool, since it makes sure that the base is solid, rich enough and already enables people to create things out of the box. The latter has the obvious benefit that it integrates with existing tools, attracts people who are familiar with the technology and also already provides one of the plugins. Again, which one, though? You will always turn away some groups ☹

dumblob commented 3 years ago

I'd second what @minecrawler wrote above. There is one important omission in (sure, it wasn't meant to be exhaustive but anyway :wink:):

Which is why I would suggest to go and spike the issue. What do all of the paths have in common conceptually? What kind of properties do they set and how does layouting work? What happens when multiple screen proportions (4:3 vs 16:9 for example, but also mobile vs tablet and similar) and resolutions (1280 x 720 vs 2560 x 1440 and similar) have to be targeted? It may be a good idea to create a comparison matrix and have it filled out by different developers. Make it viewable to everyone and foster discussions, so that modern best-practices are used. Then come up with a base for them all (leaving out their respective golden kitchen sinks and special features) and set that as the bevy internal standard.

Namely the "composition in time" (the paragraph above discusses composition in space which is usually less of an issue). Composition in time is less visible, but much more aggressive and strict when it comes to options users/devs will get when using Bevy.

Please read https://potocpav.github.io/programming/2020/05/01/designing-a-gui-framework.html to understand "composition in time" and why it's much bigger issue than "composition in space". A good test case for "composition in time" could be this: https://github.com/vlang/ui/issues/7#issuecomment-576256637 .

Btw. don't get the impression after reading the blog post, that "composition in time" requires immediate mode UIs. It's fully agnostic despite the author of that blog post uses an immediate mode library as backend for his UI library.

johannesvollmer commented 3 years ago

While I don't disagree with your general point, I'm pretty sure there are plenty of people that have enough experience with CSS to see the real disadvantages.

I don't see any argument against CSS if and only if it is implementated as an opt-in abstraction, and not as the core mechanic. Why shouldn't you have both?

minecrawler commented 3 years ago

I'm pretty sure there are plenty of people that have enough experience with CSS to see the real disadvantages.

I don't think you have to look too far away. I do work with CSS in small and big applications in several professional corporate teams every day and even hold trainings. I know that CSS has its pros and cons, just like every other technology out there. However this discussion should not go into an emotional direction why one tech is bad. We want to create a UI for bevy, so we should be problem-oriented. Many people prefer CSS, many people don't like it. Let's roll with that and find a solution to make them all happy 🙂

"composition in time"

That's a very interesting topic which we have not touched, yet. It's distinctively different than to decide if we want to go web or not. However, I also fear that there are multiple architectures possible.

While I am not sure I fully grasp how Concur works, I am not entirely a fan of it (re-creating elements on change sounds inefficient, especially animations might be very scary). The author also says that they are mostly viable with retained mode libraries like React (the VDOM only propagates what really changed to the browser, so it shouldn't recreate the entire button) and immediate mode UI like ImGui, which is light-weight and is entirely recreated on each draw anyway.

What I'd like to see, though, would be the usage of the ECS. Would it for example be possible to wire certain widgets into it by giving them a certain query, which provides them with the right data at the right time? A health bar could get a query, which delivers it the current and max values (plus styling) onChange, making the ECS the single source of truth.

However, it is another point which we should talk about. Is it possible to abstract it so that we can package plugins (for this case, I doubt it, but may be wrong), or do we need to choose one? I'd very much like to hear more concepts :)

dumblob commented 3 years ago

"composition in time"

... What I'd like to see, though, would be the usage of the ECS. Would it for example be possible to wire certain widgets into it by giving them a certain query, which provides them with the right data at the right time? A health bar could get a query, which delivers it the current and max values (plus styling) onChange, making the ECS the single source of truth.

Yep. I'd very much like to see this as well. It's basically a sort of pub-sub pattern twice in a row - input sources (timer, ECS button component click, keyboard press, ...) publish to those "brokers" who are interested in them (subscribed to the given topic in pub-sub terms) and then these notified "brokers" publish to those (e.g. ECS) components interested in these (and change the health bar in reaction to those).

Finding a neat syntax for this in a "structured programming language" appears to be nontrivial (basically in a no concurrency environment this can be easily implemented using gotos, but elsewhere I can think of only heavyweight structures like Go channels which still do not provide the syntactical simplicity one would expect from such a pattern).

However, it is another point which we should talk about. Is it possible to abstract it so that we can package plugins (for this case, I doubt it, but may be wrong), or do we need to choose one? I'd very much like to hear more concepts :)

I think the orthogonal distinction "composition in space" and "composition in time" holds for any "concept". The question is how to implement this in existing languages - and here I expect many answers :wink: (just read the whole thread https://github.com/vlang/ui/issues/7# incl. all linked threads recursively to get "shocked").

IDN maybe it's time to rethink "structured programming" on it's own and revise it by addition of such a "double pub-sub" construct? Mech lang does it (though it's a bit of an extreme - basically all "variables" are "live" by default which is inefficient if the frequency with which the variable shall change is lower than the cost of bookkeeping it requires).

Pauan commented 3 years ago

If we're discussing composition of values over time, that's exactly what FRP is designed for. I have already created a robust, production ready, feature-complete, thread-safe, zero-cost ultra lightweight FRP library for Rust:

https://crates.io/crates/futures-signals

I have used it to successfully create one of the fastest DOM frameworks in the world, though of course the FRP library is agnostic so it can be used for anything (not just the DOM).

Since Bevy already uses Futures internally, it is trivial to integrate futures-signals (since it is built on top of Futures, and integrates perfectly with Futures).

But integrating it with the ECS sounds rather difficult, since they are fundamentally different paradigms.

dumblob commented 3 years ago

But integrating it with the ECS sounds rather difficult, since they are fundamentally different paradigms.

That's exactly what I said above: The question is how to implement this in existing languages - and here I expect many answers :wink:.

Though I'm almost sure the "double pub-sub" concept could fit ECS pretty well (efficiently and readably/understandably). I'm experimenting with it a bit now (in Python though) and if I find something I'll come back at some point.

Btw. futures-signals are awesome - I'd be quite interested in your thoughts @Pauan why integrating them with ECS sounds rather difficult (but maybe we should discuss this elsewhere).

Edit: I made a separate topic about "ECS versus signal-slot/pub-sub" - see https://github.com/Pauan/rust-signals/discussions/31 .

Krahos commented 3 years ago

Concerning UI, I've been working with WPF (XAML), Xamarin, Unity, Unreal and Flutter both as solo developer and in small teams with graphic designers. Strictly speaking about developer experience, I prefer by far Flutter's way of doing things: using the same language to both write the logic and the UI and decoupling the two with the BLOC pattern and with streams. On the other hand, I saw my colleagues who don't know anything about programming having a much more pleasant experience using Unreal's blueprints. For Bevy, if it's aimed to be "by developers for developers", I cannot recommend enough to learn from Flutter, at least initially and then, if technically possible, integrating a visual designer to allow artists to design things by "dragging and dropping" widgets.

erikpukinskis commented 3 years ago

I've been a webdev for about 25 years. One thing that's become clear to me over the last few years: React was a good idea.

I've recently started playing with Bevy too. I've made lots of toy games over the years, often doing as much as I can from scratch. But lately I want to just use good, solid tools. I'm trying harder to not re-invent the wheel. I totally support anyone who wants to build a brand new FRP UI library for Bevy, but in my mind, there are already great UI building libraries out there, so why not stand on the shoulders of giants?

So, in the name of not reinventing the wheel, I would like to use React for my game UI. Doesn't look like it's possible to render a browser page into Bevy right now, but that got me thinking...

React actually already has a way of building apps that target platforms other than the web: React Native. And there are a number of targets ("platforms") that various non-Facebook people are maintaining.

It's apparently not easy, but making it easier to maintain a React Native platform is one of the major goals of their next rewrite: React Native Fabric. It seems to be deep in early alpha territory at the moment, but that might not prevent at least doing some research on it.

Perhaps the most straightforward implementation of React Native in Bevy would be to fire up a V8 instance, build your mini apps in JavaScript, and then just build the React Native rendering service that ties them into Bevy. I suspect one could do so iteratively, building out the various modules and attributes as you needed them.

Another approach would be to just adopt the React Native API, but to keep everything in Rust. That way you can ditch the JavaScript runtime. You would have some guarantees of having a robust, familiar API design but no real constraints on how it would be implemented or integrated. Still, at that point it seems like you might as well just design something from the ground up that's as Rust-y as possible.

Pauan commented 3 years ago

@erikpukinskis Note that there are a lot of quirks with React, it's a complex beast. And it was also designed specifically for JS, it would be difficult to integrate it with Rust.

The idea of React is indeed quite good, it's essentially an immediate mode GUI, which is a lot nicer than the DOM (which is retained mode). However, we can gain the advantages of immediate mode GUI without React.

A Bevy-specific UI can be much lighter-weight, more intuitive, more Rust-y, have better integration with ECS, and have more features.

ShalokShalom commented 3 years ago

I've been a webdev for about 25 years. One thing that's become clear to me over the last few years: React was a good idea.

I've recently started playing with Bevy too. I've made lots of toy games over the years, often doing as much as I can from scratch. But lately I want to just use good, solid tools. I'm trying harder to not re-invent the wheel. I totally support anyone who wants to build a brand new FRP UI library for Bevy, but in my mind, there are already great UI building libraries out there, so why not stand on the shoulders of giants?

So, in the name of not reinventing the wheel, I would like to use React for my game UI. Doesn't look like it's possible to render a browser page into Bevy right now, but that got me thinking...

React actually already has a way of building apps that target platforms other than the web: React Native. And there are a number of targets ("platforms") that various non-Facebook people are maintaining.

It's apparently not easy, but making it easier to maintain a React Native platform is one of the major goals of their next rewrite: React Native Fabric. It seems to be deep in early alpha territory at the moment, but that might not prevent at least doing some research on it.

Perhaps the most straightforward implementation of React Native in Bevy would be to fire up a V8 instance, build your mini apps in JavaScript, and then just build the React Native rendering service that ties them into Bevy. I suspect one could do so iteratively, building out the various modules and attributes as you needed them.

Another approach would be to just adopt the React Native API, but to keep everything in Rust. That way you can ditch the JavaScript runtime. You would have some guarantees of having a robust, familiar API design but no real constraints on how it would be implemented or integrated. Still, at that point it seems like you might as well just design something from the ground up that's as Rust-y as possible.

Just a heads up:

Elm made FRP popular and inspired React directly. React only went back to Javascript for the obvious reasons.

Elm then quickly realized that their own, new MVU based "Elm architecture" has couple of significant improvements compared to FRP, and dropped that one already 4 years ago or something.

dasifefe commented 3 years ago

With the mockup that I am playing around with in https://github.com/bevyengine/bevy/issues/85#issuecomment-868103713, I took inspiration mainly in the visuals of VS Code. Flat UI and only the colors are theme-able, what users would distribute to each other would be color themes (palettes), not CSS themes.

This would prevent all the work on a CSS-based architecture, which probably would be only a subset of CSS. With my experience with GTK3, CSS themes might be very flexible, but I have never seen the GTK community fully satisfied with it, breakages of themes often happens with GTK updates. Now, consider that GTK/Gnome development community is way, way bigger and many years older than the Bevy community.

In my opinion, for the editor specifically, the focus should be on: 1) DPI scaling, 2) clean look and 3) color theming. For the use of the UI outside the editor, then explore all the way with theming.

mrDIMAS commented 3 years ago

Hello! How about rg3d-ui? I've successfully making editor for rg3d engine with it. IMO it has almost everything that is needed for the editor, including docking, windows, dropdown lists, text boxes, trees, file browsers, and even specific widgets that allows editing Vector3 and so on. It can be easily integrated in bevy - all you need to do is to write a renderer (rg3d-ui knows nothing about rendering on screen) and feed it with window events.

mjoork commented 3 years ago

I am not a game engine developer of any kind and not a GUI person either, but this talk seems very relevant to the problem.

Leaving my 50 cents so to say.

https://youtu.be/yYq_dviv1B0

EDIT: Especially more towards the end >40 min or sth

Kleptine commented 3 years ago

Great talk -- the Our Machinery people are really smart folks.

I will say that while IMGUI works for smaller engines (and may even work for Bevy!), Unity started with an IMGUI approach and has since spent the last 4 years attempting to migrate off of it, for performance reasons. I think it's great for some basic UI, but performance suffers for more complicated layouts.