Option to "trap" the cursor inside the fullscreen window

[hpohl]

Especially for strategy games it would be useful to keep the cursor inside the fullscreen window to not lose focus when using multiple screens.

The Pointer Lock API has a use case for this: https://w3c.github.io/pointerlock/#xview-port-panning-by-moving-a-mouse-cursor-against-the-bounds-of-a-view-port. This would involve letting the game render the cursor by itself, introducing latency which is unacceptable for gaming.

[annevk]

Why can't Pointer Lock be used for this?

cc @scheib

[hpohl]

Using Pointer Lock would hide the no-latency system cursor.

[annevk]

I see, so you want Pointer Lock, but keep the system cursor. It seems like the easiest solution would be to add an option to Pointer Lock to not hide the system cursor, but I might still not fully understand.

[hpohl]

I was not fully sure myself whether to post this here or at the Pointer Lock repo. It's somewhere in between. From my perspective it's closer to the Fullscreen API. In the FAQ section (https://w3c.github.io/pointerlock/#why-bundle) they point out a possible solution for trapping the cursor inside any DOM element and providing a more fine grained control, but due to technical issues this will not be implemented yet, which I understand. It's also a way more generic approach compared to what is needed.

It's a very common feature every RTS-like game has which the browser is lacking at the moment.

[annevk]

Okay, so yeah, if it's too expensive to enable generally, but doable fullscreen, we could have a flag of sorts passed to requestFullscreen() (which could get a little icky to feature detect, we should make sure that's possible). If we did it that way we'd still have to meet the Pointer Lock requirements of course around sandboxing and such so Pointer Lock would have to be refactored to make some of its algorithms and checks reusable by Fullscreen.

Alternatively, we could keep it separate and make it an argument to the Pointer Lock API that ends up rejecting if you're not fullscreen. That would be a more modest change and not make Fullscreen dependent on Pointer Lock (though the reverse would be true, albeit in a much less invasive way).

[hpohl]

I would vote for the former which is adding an optional parameter to requestFullscreen().

[hpohl]

Also I don't really see how any Pointer Lock related tech would be overlapping with having it inside the Fullscreen API.

[annevk]

I don't understand what that means.

[hpohl]

What algorithms and checks of Pointer Lock need to be reused by Fullscreen?

[annevk]

The bits about it not succeeding over at https://w3c.github.io/pointerlock/#dom-element-requestpointerlock and maybe more, haven't looked in detail.

[hpohl]

I understand. It would be great to hear the opinion someone who works on Pointer Lock on this topic.

[upsuper]

I'm not quite sure whether all platforms have provided API for window-level pointer restriction. It may need some investigation into whether it is implementable in general.

On the permission side, I don't think it is problematic, though. IIRC, at least in Firefox, we allow pointer lock whenever the site is in fullscreen. The proposed API here seems to be even less harmful than the Pointer Lock API from the perspective of security (because user would still see the real pointer), so I'm not too concerned about that.

Though, it's probably still worth thinking about the API design, and whether (or how) this would interact with Pointer Lock API.

[upsuper]

Another thing may worth thinking about is what should happen for user-initiated focus change, e.g. when user uses Alt-Tab to switch to other window. I believe we want to untrap the pointer in that case. And what should happen when user switch the focus back? If the parameter is added on Fullscreen API, we probably need to re-trap the pointer in that case, then where should we put the pointer?

It's worth checking what Pointer Lock API does for this situation. But since Pointer Lock API can hide the pointer, where to put the pointer is probably not a question it needs to consider.

[hpohl]

Regarding the implementation: Cross-platform RTS-like games like HoN and Dota 2 have been doing it for years on Windows, macOS and Linux. I agree, security should not be an issue since we already have Pointer Lock. I'd like to think about Fullscreen and Pointer Lock as two separate things, so even if you are already fullscreen and trap the cursor, you can still lock it on another element.

AFAIK, when you tab out, you lose pointer lock which makes sense. It would be nice to automatically re-trap the cursor when you tab back in in this case.

[mirague]

Having this functionality would be a great step closer towards more native-like gaming experiences in the browser. I think the functionality is different from Pointer Lock, however perhaps we could re-use existing implementations for preventing the cursor from leaving the currently fullscreen'ed element.

When Pointer Lock is requested in "Fullscreen Trap" mode, we could hide the cursor exactly how it behaves nowadays. When Pointer Lock is exited (programmatically) it would be great if the cursor trap is still active. This would allow for toggling Pointer Lock on and off (a likely use case for a game).

[scheib]

This is an old feature request becoming more relevant now that there are more implementations of Pointer Lock. It's worth revisiting, but needs to be prioritized against other input work.

I called it "Pointer Clip" in Pointer Lock's Features to be considered for Next Version. It wasn't acted on earlier because a) opportunity cost -- just didn't out prioritize other work and b) there is a workaround, Pointer Lock with an application drawn cursor does work, but I certainly grant that the lag makes for a poorer experience.

The implementation complexity was, (from memory 6 years ago), primarily around arbitrary rectangles on Linux systems. Especially with possibility of moving windows, this was harder. Also, note that compared to games which typically lock the entire window, we may want a web API that locks to a DOM element and works like MSs ClipCursor

Limiting to the fullscreen case would be simpler, but I think a 'local maxima' of a feature. Plenty of games and other applications would benefit from a) being able to lock when not fullscreen and b) locking cursor to only a portion of the screen.

cc @dtapuska

[hpohl]

Not sure how I missed that document, thank you for the insight. Needless to say, I am interested in revisiting it in any form. If there is anything I can do, let me know.

[scheib]

To move forward I'd say we need to gather interest from at least two browser's input teams and see them rank this as something to work on. There is an upcoming TPAC session where I will bring this up.

[annevk]

I'm pretty sure Mozilla would be interested in this one way or another. We want the web to be great for games.

[mirague]

That sounds like a great initiative, thanks for carrying this forwards @scheib (also for your efforts in the past, I've read your notes). Hopefully some browser input teams will pick up on this and share their thoughts on this matter.