Moving the mouse outside the game or preview window trigger left/right/middle clicks

KinkGD commented 5 years ago

Describe the bug

When moving the mouse cursor outside the preview window, or outside game window when it's windowed ( windows build/export), it triggers left/right and middle mouse click/release.

It's a bit annoying because if you use events that calls some actions on these conditions, moving the mouse outside the window calls the events..

To Reproduce

Steps to reproduce the behavior:

New project : new object => add event to create object at MouseX;mouseY coords (triggering on left or right or middle mouse button realease)
Preview or export to an executable (tried only on windows here)
Move the mouse outside the window, and see thaht an object is created..

I can make an example, but i suppose it's already known... The problem is that it "breaks" some gameplay if playing in windowed mode. I used a workaround until now (conditions to avoid triggering clicks if the mouse is outside window scene (borders -1px)), so i've forgot a bit about the issue.. but i constated it's still here while playing with middle mouse button fix.

Other details

Windows 10 x64, GD5b62 and previous.

4ian commented 5 years ago

It's actually a "feature", the game engine specifically trigger the release of the button if you go out of the screen. I think it was to avoid things like:

You press the left button
You goes out of the screen
You release the button BUT the game is not aware of this event.
You put back your mouse in the game, and you have now the button released but considered as pressed by your game

zatsme commented 5 years ago

Sounds like we need a preference in options on how to handle mouse leaving the window, then the game designer can choose which is best for their game?

ghost commented 5 years ago

I've been having this issue, but it happens whenever the mouse leaves the window regardless of whether the button is held down or not.

@4ian suggests that it's a feature to prevent weird things happening when the button is released outside the game (which I understand), but based on my experience (see attached example) and @KinkGD's report, it seems to be happening when the mouse leaves the window without the button being held down first (which seems more like a bug and less like a feature)...

In the attached example, the each player's score is updated when you click near them (I built this to test http://forum.compilgames.net/viewtopic.php?p=70081#p70081) however I noticed that when I moved the mouse outside the window to click the close button, the green player's score was updated. Moving the mouse in and out of the window still triggers the event that updates the closest player's score, without the mouse button being used at all. I'd be interested to know if others experience this same issue with this example (or if I just have a buggy mouse).

My project.zip

KinkGD commented 5 years ago

Sounds like we need a preference in options on how to handle mouse leaving the window, then the game designer can choose which is best for their game?

+1000, because working with several layers, and different level of zoom, i can't get any "workaround to work"... My game is totally unusable in windowed mode, and even in preview, it's painful (because we can't get real fullscreen in preview). And i suppose it's real problem for every games that use mouse button for some actions (eg : left mouse = shooting result that you could waste ammo, or shoot at something the user doesn't want to shoot (and triggering a peacefull AI).. Lot of problem..

If i take some examples from my current project (a 2d space game with rpg/roguelike features (with faction status neutral/allied/friend)) , this is what does the "mouse outside feature" :

simulate a left button release : shooting a rocket => the player shoot a rocket on friend or neutral faction ship's : he become ennemy to the faction (unwanted behavior unless it's player choice!)
Simulate a right button release : moving the player ship to the position of mouse last coords => Moving at the wrong coordinate : player ships might enter in collision with other ship, (and waste fuel in a "fast travel mode")
Simulate the middle button release : center the camera on the player ship => centering the camera on player ship : breaks the gameplay, because the player just wanted to scroll the camera on the layer, some times far frm his ship

This is really really unoptimal ui behavior.

@4ian In the "feature" like you explained it : the only problem is when you hold a mouse button outside the screen and release a mouse button, so basically, the user isn't supposed to do that. If he does that, the release shouldn't be triggered : i suggest (maybe it's not the best option) the "hold" status should be keeped and the user should have to click again somewhere in the window to reset the button holding. It's basically something i have already seen in a lot of games (but it's still make the game looks buggy according to standard). Or just remove even the "hold" status..

What do you think ? Or i totally disable windowed mode (that's the only option for my gameplay!).

EDIT : ha and even in a windows build/export and fullscreen forced : same behavior if you have dual (or more) screens ! The mouse can exit the window game to the others screens...

Wend1go commented 5 years ago

I've been having this issue, but it happens whenever the mouse leaves the window regardless of whether the button is held down or not.

I noticed this too.

Bouh commented 5 years ago

At one time or another GD simulates all mouse clicks for an unknown reason. (Tested on beta62 and other see table under this post.) See for yourself on this very simple example. When I click only left at one time a wheel click and a right click are simulate. If I click with another button the buttons that have not been clicked will still be clicked.

And then I didn't leave the game area, and the mouse didn't move!

game.zip

Bouh commented 5 years ago

All of this is tested on preview html5

		Bug click	click when mouse out of the screen	Note
GD4	4.0.97-0	NO	YES	But middle click is not supported (Left is clicked instead)
GD5	b62	YES	YES	Before this beta the middle click is on left
GD5	b61	YES	YES
GD5	b59	YES	YES
GD5	b58	YES	YES
GD5	b16	YES	YES
GD5	b15 to b2			Not tested, and some versions is missing
GD5	b1 (yes b1)	NO	YES

@KinkGD

Bouh commented 5 years ago

I found out where the unexpected clicks are made when the cursor is taken out of the screen and also the random clicks.

Look the GIF, you can see mouseOut is called by pixi.js, This is the feature you're talking about i guess @4ian .

It's actually a "feature", the game engine specifically trigger the release of the button if you go out of the screen. I think it was to avoid things like:

You press the left button

You goes out of the screen

You release the button BUT the game is not aware of this event.

You put back your mouse in the game, and you have now the button released but considered as pressed by your game

I've deleted this lines 312 to 315. And the issue with random click is over. Can we modify this code ? because random click if very annoying and impossible to prevent with events.

I've tryed with "level editor example", when i out very fast the mouse and i click but with the mouse outside the game, the game don't catch the click, and no block in game is added in the scene.

But it true if we press and we goes out and come back in-game, pressed mouse is always here. Why use simulate 3 click like a real user ? it's not possible to do in private just inside the engine ?

Example online with 3 click deleted

click_catch

Wend1go commented 5 years ago

There doesn't seem to be a check if the button has been pressed in the first place. The inputmanager just sets _pressedMouseButtons to false and _releasedMouseButtonsto true. I think when the cursor leaves the window, GD should first check if the button is pressed at all and only in this case trigger onMouseButtonReleased functions.

4ian commented 5 years ago

I've deleted this lines 312 to 315. And the issue with random click is over. Can we modify this code ? because random click if very annoying and impossible to prevent with events.

Thanks for the investigation, we can remove then these lines I think.

I think when the cursor leaves the window, GD should first check if the button is pressed at all and only in this case trigger onMouseButtonReleased functions.

Maybe too, but this can be surprising too, so maybe let's remove all together the simulation of button release

KinkGD commented 5 years ago

Should i give a try and remove these lines in a pull :

manager.onMouseButtonReleased(gdjs.InputManager.MOUSE_LEFT_BUTTON);
manager.onMouseButtonReleased(gdjs.InputManager.MOUSE_RIGHT_BUTTON);
manager.onMouseButtonReleased(gdjs.InputManager.MOUSE_MIDDLE_BUTTON);
manager.onMouseWheel(0);

From what i've tested, like @bouh does, it works : no more arbitrary left/right clicks generated when spam clicking on the scene, and no more (messy) click when the mouse is outside of the render window...

But maybe i'm missing someting important on tests ? You know that i'm very enthousiast to remove this "mouse outside click feature" and the arbitrary click is a real issue in my current point and click project (generating bugs on my dialog engine) :)

4ian commented 5 years ago

Yeah it should be fine to remove these 4 lines. Also should surely be removed in Cocos2d-JS renderer too. I don't see any bad side effects.

KinkGD commented 5 years ago

Just to be sure, it's thes lines ?

https://github.com/4ian/GDevelop/blob/8c8c297adbf38a9f751551c49e683f277407f7e5/GDJS/Runtime/cocos-renderers/runtimescene-cocos-renderer.js#L119-L121

Also, in both pixi.js and runtimescene-cocos-renderer.js, do you prefer these lines to be removed, or just set as comments ?

4ian commented 5 years ago

Just to be sure, it's thes lines ?

Sorry my mistake there is nothing is Cocos2d-JS renderer doing actually the fake button release. The lines you've shown are the actual, proper code to bind the "mouse up" events of Cocos2d-js to the input manager of the GD game engine - so it's legit.

Also, in both pixi.js and runtimescene-cocos-renderer.js, do you prefer these lines to be removed, or just set as comments ?

Remove the lines (well, only in runtimegame-pixi-renderer.js) ;) It's better to keep a code clean - and pretty sure according to your tests that it's the right solution and that these extra lines did more harm than anything else.

If you have time, you can try to make a pull request for it :) Let me know.

KinkGD commented 5 years ago

Let's doing this right now :)

Bouh commented 3 years ago

Close with PR https://github.com/4ian/GDevelop/pull/923 And tested on beta 103.

4ian / GDevelop