search

rust-windowing / winit

Window handling library in pure Rust
https://docs.rs/winit/
Apache License 2.0
4.73k stars 888 forks source link

The case for callbacks #2010

Open madsmtm opened 3 years ago

madsmtm commented 3 years ago

Hey, I've been tinkering with (primarily) the macOS backend for a while now, and I feel like I've been bit several times by the callback-y nature of AppKit poorly matching the (almost) data-only Event.

So here's my proposal for remedying this.

Background

Most of the underlying system APIs work by letting the user register callbacks on a window, that then get called when an event happens. See the below table for a complete overview:

Library Method
AppKit Multiple callbacks per application/window/view class.
UIKit Same as AppKit.
Windows Single callback per window class.
Web Multiple callbacks per DOM element.
Wayland Multiple callbacks on "seats" (don't really understand this).
X11 Uses an internal event queue that you poll from.
Android Same as X11.
Orbital Same as X11.

winit then turns this into a variant of event::Event, and either calls the event handler directly or buffers it for later (currently buffered events on macOS: WindowEvent except ScaleFactorChanged, DeviceEvent and UserEvent).

The issue

Well, this is honestly a pretty good user-facing API! It's easy to share data between event handlers (since there's only one), and is in general pretty "rusty".

However, it's very much an abstraction, and as with almost any abstraction, we lose some form of control.

One problem is that it's hard for users to know which events are emitted directly / blocking, and hence need to be acted on immediately (like RedrawRequested), and which ones are buffered and can be handled at leisure.

Another is that the user can't directly respond to the events unless we break the assumption that Events are just data, see https://github.com/rust-windowing/winit/issues/1387.

Thirdly, on macOS, the behaviour is different depending on whether a callback is registered or not, see https://github.com/rust-windowing/winit/pull/1759#discussion_r571592243. So we can't expose this behaviour as an event, because it would negatively affect users that aren't handling the event.

In general, the design forces you to pay for stuff you don't use, which is against the Rust principle of zero-cost abstractions.

The actual proposal

I would like to propose that we change the internals of our backends into having callback-style APIs.

We'd still build the much more user-friendly EventLoop API on top of that, but we'd have the goal of at some point in the future exposing some of these "lower-level" callbacks to user code.

Apart from being a way forward on fixing the above issues, I think it would improve our backend's code quality; one part of the backend would focus on making a safe abstraction (with all the Send and Sync that entails) that matches the OS' callback-based nature, while another could focus on the order events are buffered and how they're dispatched.

Prior art

Future possibilities

We might be able to split the lower-level callback part of winit off into a separate crate or something (probably still same repository), which might be able to become a shared base between winit and druid-shell? In any case, we should have a way forward on this.

Finishing up

I know that what I'm proposing is somewhat vauge here, please bear with me. If people are on board with this, I'll try (at some point) to spearhead with the macOS implementation, then the benefits might become more apparent.

msiglreith commented 3 years ago

Interesting proposal, I'm not totally sure where this is heading API-wise but I'm not opposed to it in general

Another is that the user can't directly respond to the events unless we break the assumption that Events are just data, see #1387. Thirdly, on macOS, the behaviour is different depending on whether a callback is registered or not, see #1759 (comment). So we can't expose this behaviour as an event, because it would negatively affect users that aren't handling the event.

There is a similar situation on Android where we have to mark certain events as 'handled' or not for the OS (e.g volume buttons) which would require additional return values from users.

rib commented 2 years ago

It would be good to understand a bit more specifically what problem the proposal is trying to solve, to be able to judge the trade offs more clearly.

To some extent the current winit event loop 2.0 design is callback based; it's just that you have a single uber callback. For comparison a more pure data-oriented event system (i.e. non-callback-based design) in rust would probably be a stream of events, which isn't really the case currently.

Considering the Android backend; the fact that the current design of Winit backends is callback based is actually quite important because it means we can transparently handle synchronization with Java in places where we need to - which wouldn't be possible to support if Winit wasn't already callback based.

Put another way; I think it's maybe not so much about being callback based or not that's being raised here - it's about proposing a more fine-grained callback system; more comparable to how delegates are registered on macOS / iOS.

Each window system has its own quirks though and although modelling the design more on macOS / iOS would probably make things neater for their respective backends, it might also complicate things for other backends - so it could be good to consider the details a bit more to understand the specific problems.

One issue with the current design seems to be that it's not obvious how backends can extend the events that may be emitted to be able to expose certain platform-specific details in situations where it's not really appropriate for Winit to abstract those things (re: #2120)

A general concern I'd have with fragmenting the callbacks though would be with defining clear locking and re-entrancy guarantees. If you have ad-hoc callbacks then what guarantees (if any) would you get about what thread they run on for all OSs, and what actions can safely be performed in each callback that won't either trigger re-entrancy or invoke other callbacks that might potentially need to access the same shared state (and lead to dead locks if not careful).

Having a single point of entry for events / callbacks can help remove some of those concerns / complexities.

From what I recall of working with macOS in the past though, then I do recall that delegates were expected to handle their responsibilities immediately which can sometimes preclude any kind of buffering (you can't just translate all delegate callbacks into an event that is handled asynchronously the next time the event loop runs).

That has some similarity with a few things on Android (e.g. handling saving state or SurfaceView termination needs to be handled synchronously in Rust once it has been notified of them) but luckily on Android those cases can still be handled fine with the current Winit design because those things are effectively just buffered before they are delivered to Rust.

Maybe the existing uber callback design is still mostly ok for Winit but what might be good evolve is the protocol around how each even loop iteration needs to start with NewEvents and then follow a particular order for dispatching other events. I guess on macOS you want to be able to invoke the callback in a more ad-hoc way such as for redraw events, and maybe that's awkward if you're expected to follow that full NewEvents sequence for each callback?