Mouse enter/leave window events

[abonander]

AFAICT, we don't currently provide events for when the mouse enters/leaves the window (a separate event from focus gained/lost), which would help situations like https://github.com/PistonDevelopers/drag_controller/issues/20.

Obviously, since this would be a pretty widespread breaking change, it'd be a good idea to bikeshed the design first. Perhaps add one or two new variants to input::Motion, since it concerns movement of the mouse cursor:

pub enum Motion {
    // Existing variants omitted for brevity.
    /// The mouse has entered the window area.
    MouseEnter,
    /// The mouse has left the window area.
    MouseLeave,
}

Or, it could be added as a single variant to input::Input, in parallel with Focus. I'm having difficulty coming up with a short variant name that clearly describes a mouse enter/leave event as a boolean. This was the best I could do:

pub enum Input {
    // Other variants omitted
    /// Window gained or lost focus.
    Focus(bool),
    /// Mouse cursor entered or left the window area.
    Cursor(bool),
}

[bvssvni]

I like the Input::Cursor design compared to adding two variants to Motion.

There is a potential problem if a window back-end simply stops emitting mouse events when the cursor leaves the window, because Input::Cursor(false) in this case can not be known.

Alternative is to pass in window as parameter to the controller. In the case of the drag controller the situation is a little subtle as there might be two different use cases, one where the window back-end continues emitting mouse events and one where it does not. However, I think it is a good idea to think of adding these events anyway.

[abonander]

Looking at the Win API used by Glutin, it looks like we can only get messages for when the mouse leaves the window bounds. I don't know how the other APIs handle it but that's probably a good baseline for a least-common-denominator API. I guess a "mouse entered" event is implied if we start getting "mouse moved" events again after "mouse left". So something like Input::MouseLeave, perhaps with an optional coordinate pair for its last position, if available.

I think the user can stop the drag on the drag controller by watching for the MouseLeave event and setting the pub drag: bool field to false. That way they can choose how they handle the semantics of their chosen window back-end. If that's all right with you, PistonDevelopers/drag_controller#20 can be closed.

[bvssvni]

Could not find a similar event for SDL2.

[abonander]

Wouldn't it be SDL_WINDOWEVENT_LEAVE?

[bvssvni]

@cybergeek94 You're right. It's here too http://angrylawyer.github.io/rust-sdl2/sdl2/event/enum.WindowEventId.html

[bvssvni]

Testing with the user_input example from piston-examples (OSX):

• Glutin continues events outside the window
• SDL2 stops sending events outside the window
• GLFW continues sending events outside the window

[abonander]

Intriguing. I'm using Glutin in https://github.com/cybergeek94/logic-circuits and mouse events stop outside the window area for me. This is on Linux and Windows.

[bvssvni]

If we restrict the semantics of the mouse leave event to interrupt dragging, then we could have an event for "interrupt dragging" which could be combining defocus and mouse leave in the current back-ends. It could take a cause as argument so controllers can choose whether they want to interrupt because of that specific event.

[abonander]

I'd probably rather just get the mouse-leave event directly. Interrupting a drag is just an example use-case.

[bvssvni]

I agree. Just thinking about alternatives. Adding mouse leave events seems the best way for now:

• If the underlying API of a window back-end does not have mouse leave events, but supports mouse movements outside the window, it could watch mouse movements and generate mouse leave events manually.
• There is no good way to handle this by watching mouse movements alone, since SDL2 stops sending coordinates when the mouse leaves the window.
• The only problem is when a window back-end does not send mouse coordinates outside the window and neither reports mouse leave events. In that case, the underlying API is probably not well designed.

From this perspective, mouse leave events seems sensible.

What remains is finding out whether we should allow mouse enter events to be implemented optionally by window back-ends.

[bvssvni]

• If the underlying API of a window back-end supports mouse leave events, then it probably supports mouse enter events (Glutin might add this in the future).
• If the underlying API of a window back-end does not have mouse leave events, but supports coordinates outside the window, the mouse enter event can be generated manually.

We could generate mouse enter events manually for Glutin when coordinates are supported outside the window, but this won't work today for Linux and Windows. We can detect whether a coordinate is received outside the window, then generate a mouse enter event manually if this does not occur next time we receive a mouse coordinate.

[bvssvni]

In summary:

The only problem is when a window back-end does not send mouse coordinates outside the window and neither reports mouse leave events. In that case, the underlying API is probably not well designed.

In this case we could use hack that allows you to trigger mouse leave events and then reenter manually by pressing a button specified by WindowSettings. This will make it possible for a user to interrupt drag using the right mouse button or a keyboard button. Not an elegant solution, but it makes it possible to recover from a state that might force the user into an undesirable behavior.

[bvssvni]

A similar hack, if expected to be supported without causing errors in all applications, could actually be used to emulate a mouse cursor using a touch screen. Each time you press a finger down, a mouse enter event is triggered, followed by a single mouse coordinate, then left button press, then mouse coordinates, then left button release, then mouse leave. A touch has the property that it can start and end in the middle of the window. This is also true if you want to trigger reenter manually.

We could solve two problems (maybe three) by adding mouse enter and leave events:

1. Support for platforms that uses touch screens as primary pointing device.
2. A way to recover from buggy behavior in window APIs or applications when drag is not interrupted properly.

[bvssvni]

The third problem is ergonomics in painting software. Painting is used very frequently in tools for gamedev so that's a thing we should look into. A way to emulate mouse reenter and left mouse button press if it was previously pressed. You can start a new stroke without lifting the primary finger and it will start from the exact spot where the previous stroke ended. On touch devices you could tap a second finger. This is out of scope for now, but it is something to think about.

[bvssvni]

I think that we should add enter/leave events. I opened up a new issue https://github.com/PistonDevelopers/piston/issues/961 and added @cybergeek94 as owner to a bunch of libraries on crates.io. Tell me if I've missed someone.

I'll write up the design decision in another issue.

[bvssvni]

I wrote about the design here https://github.com/PistonDevelopers/piston/issues/962

[abonander]

Close in favor of #961 & #962?

[bvssvni]

Ok. Closing.