search

SecondHalfGames / yakui

yakui is a declarative Rust UI library for games
Apache License 2.0
238 stars 21 forks source link

Bevy Integration #78

Open Cifram opened 2 years ago

Cifram commented 2 years ago

Bevy is currently by far the most popular game engine in Rust, and for good reason. It's a solid, well engineered design, even if it's still in fairly early development.

I looked into integrating Yakui with Bevy. I believe it's possible, because Yakui is not particularly opinionated about how it's used, but it's definitely not trivial, which is why I didn't complete the task myself. However, I'd still be very interested in taking advantage of the integration if anybody did manage it.

The Yakui object should be a Bevy resource. Because it's not Send, it needs to be inserted with insert_non_send_resource() instead of insert_resource(), and then accessed with NonSend<Yakui> instead of Res<Yakui>. There should be a plugin added which inserts this resource, and any systems or other resources needed to initialize and render Yakui.

Then it's on the developer using Yakui with Bevy to add their own system which requests NonSend<Yakui> and calls start() and finish() on it, and any appropriate UI code between those.

First issue: Extract Phase

Bevy's render pipeline has an extract phase, which copies all data needed for rendering from the main world to the render world. This allows rendering of the current frame to happen in parallel with updating the next frame. Yakui also supports an extract step, as the Yakui::paint() function returns a PaintDom, which is the extracted data needed for rendering the UI.

The problem is that the PaintDom it returns is actually just a reference to a field on Yakui. This means it has complicated lifetime semantics. It can't be movied, and can't be stored anywhere. But in order to utilize the extract phase in Bevy, it needs to be stored on a resource. This means the PaintDom needs to be cloned. It does not currently implement Clone, and I'm not sure whether cloning it is safe, and it's almost certainly not efficient.

Second Issue: Rendering

Bevy and Yakui both use WGPU, so that should make things simpler. But, WGPU is a fairly complicated API, Bevy's render system is fairly complicated, and Yakui's render system is also fairly complicated. Making these all work together is going to require a deep understanding of all three, and I suspect some refactors of how Yakui's render system is divided up.

Alternate Approach: Pre-Rendering

Alternately, rather than trying to tie Yakui directly into Bevy's rendering system, Yakui could be made to render into a buffer of some kind, and then send that to Bevy's render pipeline through nomal channels. Ideally this would mean rendering with Wgpu into a GPU texture, but I'm not actually sure how to then tell Bevy to render that as an overlay. Alternately, it could not use YakuiWgpu, and instead use something that renders it in software and just attach the resulting buffer to a Bevy sprite, but this is probably a lot slower.

Ralith commented 1 year ago

The problem is that the PaintDom it returns is actually just a reference to a field on Yakui.

A natural solution to this would be to perform the work that traverses a PaintDom to produce backend-specific rendering work in the extract phase. For example, each yakui Texture and PaintCall could be translated into analogous bevy objects.