Keys and mouse presses getting 'stuck' on mac

[vibrant]

So I'm running Catalina 10.15.3 and when I start my game and start moving around and shooting, and get in some skirmish and things get slow - eventually key and mouse presses get 'stuck' and the game assumes that a given key is constantly pressed even though it's released.

I use key.KeyStateHandler and check if a key is pressed there. I tried to use set_exclusive_keyboard() which doesn't seem to fix it.

Is this a common issue on macs or this shouldn't be happening?

[benmoran56]

Sorry for the slow reply, and thanks for opening the ticket.

This shouldn't be happening, unless somehow OS events are being dropped. You mentioned it happens when you experience slowdown in your game. With that in mind, would it be possible to make a small code example that shows the issue? If it only happens when slowdown occurs, maybe using time.sleep or some other blocking function could be used to simulate it.

I don't have access to a Mac at the moment myself, but if we can replicate the issue it will help getting it addressed.

[pglass]

Adding to this, I'm on OS X (10.15.7) with Python 3.9 and pyglet==1.5.13 and mouse events seem to frequently get stuck. This is on a 2019 Macbook Air, using the trackpad, fwiw.

I created the following minimal example in which I subclass pyglet.window.Window and override the relevant mouse event methods. It prints a log statement to stdout for each mouse event, and renders a white circle whose position is updated by the on_mouse_drag event. Let me know if I'm doing something obviously wrong:

import logging

import pyglet

LOG = logging.getLogger(__name__)

class Game(pyglet.window.Window):

    def __init__(self):
        super().__init__(width=600, height=400)
        self.title = "Mouse test"

        self.batch = pyglet.graphics.Batch()
        self.shape = pyglet.shapes.Circle(
            x=self.width // 2,
            y=self.height // 2,
            radius=20,
            color=(200, 200, 200),
            batch=self.batch,
        )

    def on_draw(self):
        self.clear()
        self.batch.draw()

    def on_mouse_press(self, x, y, button, modifiers):
        LOG.debug('on_mouse_press(x=%s, y=%s, button=%s, modifiers=%s)',
                  x, y, button, modifiers)

    def on_mouse_release(self, x, y, button, modifiers):
        LOG.debug('on_mouse_release(x=%s, y=%s, button=%s, modifiers=%s)',
                  x, y, button, modifiers)

    def on_mouse_drag(self, x, y, dx, dy, buttons, modifiers):
        LOG.debug(
            'on_mouse_drag(x=%s, y=%s, dx=%s, dy=%s, buttons=%s, modifiers=%s)',
            x, y, dx, dy, buttons, modifiers
        )
        self.shape.x = x
        self.shape.y = y

def main():
    logging.basicConfig(level=logging.DEBUG)
    game = Game()
    pyglet.app.run()

if __name__ == '__main__':
    main()

I was trying to figure out the best way to demonstrate the issue... OS X supports screen recording (CMD + Shift + 5), and there's an option to "Show Mouse Clicks". This will show a circle around the mouse pointer in the recorded video whenever the mouse button is pressed, so we can see the difference between actual clicks and Pyglet's on_mouse_* handlers/events.

https://user-images.githubusercontent.com/1077740/103391299-643f5780-4ade-11eb-86ff-2a1307e8c742.mov

(Sorry for the poor quality. Recommend viewing in full screen. I had to get under the 10mb video limit in Github, and there were limited video encoding options out of the box in OS X. I can make this better next time).

Explanation:

• On the left of the video is my terminal. I ran PYTHONUNBUFFERED=1 python mouse-test.py, which should avoid stdout buffering delays in the terminal. On the right is the Pyglet window.
• When I click, an empty circle shows around the mouse pointer. This comes from the OS X screen capture software and represents an actual mouse click.
• The solid white circle is rendered in Pyglet. This is a visual representation of the mouse drag events the Window is actually receiving and responding to.
• At the start of the video, things seem fine. I can press/release the mouse button and see the corresponding log statements show in the terminal. Dragging is also working well.
• Towards the end of the video, mouse drag events and mouse press/release events are not working reliably. I click several times and do not see an equal number of press/release log statements in the terminal. Dragging is also not working well.

Hopefully that helps. Let me know if I'm doing anything wrong, and happy to help test/retest on my Mac.

Thanks for all the work on Pyglet, as well!

[rebane2001]

Happening to me too!

When I first launch my game, everything feels very responsive and all the click events are coming through correctly:

Then, after doing some mouse stuff (doesn't seem to happen if no input) for a bit, the input gets laggy, and eventually starts dropping events.

Here you can see events happening in a way that should be impossible, because some of them are getting dropped:

@pglass already provided plenty information, so I'm keeping my comment short, but if you need any info or testing I'll be glad to help!

[rebane2001]

I did a lot more testing in the meanwhile, and while I didn't find a solution, I did gather a bit more info.

It seems like the OS-level events are getting dropped, not just pyglet events.

After pyglet enters this "unresponsive" state, the titlebar appears frozen (hovering over the titlebar buttons does not show the glyphs as it should). Memory and cpu usage appears fairly normal, and the frame rate of the game and rate of the mouse-move events seems normal.

Apart from restarting pyglet, I found two ways of getting out of this state temporarily. One is opening a menu on the top of the screen (eg the apple logo one), and the other one is clicking the maximize button to go fullscreen. Note that going fullscreen by calling window.set_fullscreen(True) does not fix the issue.

Another interesting thing I noticed is that if you move the window in the unresponsive state, you sometimes get the "Warning: Window move completed without beginning" message from the OS.

[einarf]

I can't reproduce this on my 2013 MacBook (Catalina 10.15.7)

Can be related to #225 and #575

I'm wondering what mac models people have. I know there was some major issue with the touchbar in the past because it added some weird events we could not process. Might still be worth testing ObjCInstance cache as a WeakValueDictionary to see if that is the problem.

[caffeinepills]

It looks like there is a DeallocationObserver_Implementation that is suppose to handle removing the cached objects. Perhaps something is broken there or needs to be updated?

https://github.com/pyglet/pyglet/blob/master/pyglet/libs/darwin/cocoapy/runtime.py#L1189

[rebane2001]

I'm using a 2020 M1 MacBook Air, running some version of Montery 12.x. I believe I am running pyglet through Rosetta2 (Intel) though.

Thanks for the suggestion caffeinepills, I'll get my Mac out in a moment and poke around in there.

[einarf]

Looks like there is tons of NSTaggedDate objects getting cached and they are never getting deallocated. I'm not sure if these are issues directly related to this, but it certainly can't be good.

[Mjtlittle]

I am also having this issue, receiving mouse drag and keyboard events at the same time. I also noticed when I hold down certain keys it pops up the diacritical tooltip, like it thinks i'm typing in a text field or something (ex. s => ßśš). Not familiar with any implementation details, just wanted to give my testimony. And I am on a 13-inch M1 2020 Macbook Pro, Monterey 12.0.1.

[rebane2001]

I wonder if this could be related to either Monterey or M1 processors?

[einarf]

I suspect there are more than one problem. There is something special needed to handle events on macbooks with touchbar specifically.

[rebane2001]

I don't have touchbar on mine, so I can confirm at least one instance of this issue is not caused by the touchbar.

[rebane2001]

I made pglass's game a bit more fun and added counters: one for how many clicks have been made so far and other for the click balance.

import logging
import pyglet

LOG = logging.getLogger(__name__)

class Game(pyglet.window.Window):
    def __init__(self):
        super().__init__(width=1280, height=720)
        self.title = "Cookie clicking game"
        self.click_count = 0
        self.mouse_down = 0

        self.batch = pyglet.graphics.Batch()
        self.cookie = pyglet.shapes.Circle(
            x=self.width // 2,
            y=self.height // 2,
            radius=192,
            color=(201, 151, 50),
            batch=self.batch,
        )

    def on_draw(self):
        self.clear()
        self.batch.draw()
        pyglet.text.Label('Click the cookie!',
                          font_size=36,
                          x=self.width//2, y=self.height - 75,
                          anchor_x='center', anchor_y='center',).draw()
        pyglet.text.Label('Your score: ' + str(self.click_count),
                          font_size=36,
                          x=self.width//2, y=50,
                          anchor_x='center', anchor_y='center',).draw()
        pyglet.text.Label('Mouse down: ' + str(self.mouse_down),
                          font_size=36,
                          x=self.width//2, y=100,
                          anchor_x='center', anchor_y='center',).draw()

    def on_mouse_press(self, x, y, button, modifiers):
        LOG.debug('on_mouse_press(x=%s, y=%s, button=%s, modifiers=%s)',
                  x, y, button, modifiers)
        self.click_count += 1
        self.mouse_down += 1
        self.cookie.radius = 192 - self.mouse_down*32

    def on_mouse_release(self, x, y, button, modifiers):
        LOG.debug('on_mouse_release(x=%s, y=%s, button=%s, modifiers=%s)',
                  x, y, button, modifiers)
        self.mouse_down -= 1
        self.cookie.radius = 192 - self.mouse_down*32

    def on_mouse_drag(self, x, y, dx, dy, buttons, modifiers):
        LOG.debug(
            'on_mouse_drag(x=%s, y=%s, dx=%s, dy=%s, buttons=%s, modifiers=%s)',
            x, y, dx, dy, buttons, modifiers
        )

def main():
    logging.basicConfig(level=logging.DEBUG)
    game = Game()
    pyglet.app.run()

if __name__ == '__main__':
    main()

On my machine (M1 MacBook Air), provided I only left click and do no dragging, it seems to get stuck after around 90-100 clicks. A friend tried it on a M1 MacBook Pro and got futher than that, but encountered the same issue.

I wonder if this is a M1, Monterey, or some other issue. Could anyone with a non-M1 Mac running Monterey (or vice-versa, M1 running pre-Monterey) give it a shot and let us know if the issue occurs.

[einarf]

Works fine for me in Catalina. The only problem is memory leaking due to the event cache filling up.

[rebane2001]

@einarf Do you have an M1 Mac?

[einarf]

Early 2013 MacBook (Intel)

[rebane2001]

Could be an M1 thing then, I have a friend who will at some point be able to test on another non-M1 Mac. It might also be worth a shot for me to try to install an Intel/Rosetta version of Python and see if the issue occurs there too.

[einarf]

This might actually not be a terribly difficult issue to fix. Debugging what is happening with the ObjCInstance instances is a good start. The _cached_objects filling up is probably a good start, but someone with an M1 would have to do this work.

Right now it seems the cache is getting filled with parent class event objects instead of the superclass since they are never released?

https://github.com/pyglet/pyglet/blob/9a1701aad79f864fa607d2c212f86df5cba37371/pyglet/libs/darwin/cocoapy/runtime.py#L932-L979

[shenjackyuanjie]

how this going?

[caffeinepills]

You can try my branch here: https://github.com/caffeinepills/pyglet/tree/macosx_dev to see if there is any change.

This doesn't fix the M1 issues, but should fix the memory leaks.

We have dug into the M1 issues and have not found a solution. We have narrowed down the issue to a particular set of function calls, but there is no reason it should not work. Until we have someone with extensive knowledge of Objective C, MacOS, Cocoa, Ctypes, Python, etc... who can do some sort of actual debugging, we probably can't resolve it.

[caffeinepills]

This should be resolved in master branch.

[jasonm23]

Please note, this does not seem to be resolved in latest, on Macos Monterey.

[benmoran56]

@jasonm23 You will need to manually enable this change for now. Set pyglet.options['osx_alt_loop'] = True Immediately after importing pyglet to try it.

[zwade]

Hi, was anyone able to root cause the m1 issue/is there a separate issue to track it? Running into this pretty consistently

[caffeinepills]

@zwade You will need a newer version of pyglet to get the newer alternate loop that bypasses the issue. Just update your pyglet to the latest. However there are a few lingering issues separate from this.

That being said, I can't replicate an issue with stuck events in the latest branch. If you have a way to replicate an issue, feel free to open a new issue with an example.

[pushfoo]

@caffeinepills which pyglet versions have the alternate event loop? Someone is reporting it on the Arcade server with pyglet==2.1.dev5. They're on an M2 with mac OS 15.0.

[caffeinepills]

There are 3 separate issues being discussed here.

1) An M1 specific issue where events are either overloading the event loop or being missed after using the program for some time. (Should be fixed with the osx_alt_loop, which is now default on arm64 MacOS)

2) An issue with MacOS data not being freed properly, leading to a memory leak. (This is resolved prior to this, and a new implementation was added to master branch)

3) When the Window is created, the focus is stolen away from another app, leaving input (mouse and keyboard) to be wonky until you minimize and restore it. The window will look like this, where the buttons are greyed out:

There are two separate issues. 1 and 2 is what this thread should be about. There is a workaround for 3, but needs more testing.

[taos]

I am seeing what I suspect is this issue, while working through the Python Arcade tutorial: https://api.arcade.academy/en/latest/tutorials/card_game/index.html. It seems to get confused after 10-30 clicks and stops registering new clicks Mac OS 14.7.1, M3 Pro

arcade==2.6.17 attrs==24.2.0 cffi==1.17.1 Pillow==9.3.0 pycparser==2.22 pyglet==2.0.dev23 pymunk==6.4.0 pytiled-parser==2.2.0 typing_extensions==4.12.2

(Note that the problem seems to be resolved in the 3.0.dev branch of Arcade, that uses pyglet==2.1rc1)

I was able to fix my issue with arcade 2.6.17 by forcing it to use pyglet 2.0.18.