NSWindow drag regions should only be invalidated on the Main Thread

[super-ibby]

Hi,

I'm using the obs-zoom-and-follow Python script with OBS on MacOS 12.6. It leverages pywinctl, and throws the following exception:

objc.error: NSInternalInconsistencyException - NSWindow drag regions should only be invalidated on the Main Thread!

The full script log is below, which may be insightful.

[zoom_and_follow_mouse.py] Source update
[zoom_and_follow_mouse.py] No sources, likely OBS startup.
[zoom_and_follow_mouse.py] Source Name: Firefox
[zoom_and_follow_mouse.py] Updating Source List
[zoom_and_follow_mouse.py] System: Darwin
[zoom_and_follow_mouse.py] browser frame | <Swig Object of type 'obs_source_t *' at 0x12f6fb6c0>
[zoom_and_follow_mouse.py] Firefox | <Swig Object of type 'obs_source_t *' at 0x12f6fba50>
[zoom_and_follow_mouse.py] logo | <Swig Object of type 'obs_source_t *' at 0x12f6fbf60>
[zoom_and_follow_mouse.py] cam frame | <Swig Object of type 'obs_source_t *' at 0x12f6fbc00>
[zoom_and_follow_mouse.py] webcam | <Swig Object of type 'obs_source_t *' at 0x12f767b40>
[zoom_and_follow_mouse.py] canon m50 | <Swig Object of type 'obs_source_t *' at 0x12f766d60>
[zoom_and_follow_mouse.py] PowerPoint | <Swig Object of type 'obs_source_t *' at 0x12f766cd0>
[zoom_and_follow_mouse.py] white | <Swig Object of type 'obs_source_t *' at 0x12f766c10>
[zoom_and_follow_mouse.py] Yeti Mic | <Swig Object of type 'obs_source_t *' at 0x12f766ee0>
[zoom_and_follow_mouse.py] New source: True
[zoom_and_follow_mouse.py] Updating Monitor List
[zoom_and_follow_mouse.py] Monitor override list updated
[zoom_and_follow_mouse.py] window_capture
[zoom_and_follow_mouse.py] Source Type: window_capture
[zoom_and_follow_mouse.py] Proceeding to resize
[zoom_and_follow_mouse.py] Updating stored dimensions to match current dimensions
[zoom_and_follow_mouse.py] /usr/local/Cellar/python@3.10/3.10.7/Frameworks/Python.framework/Versions/Current/lib/python3.10/site-packages/pywinctl/_pywinctl_macos.py:427: UninitializedDeallocWarning: leaking an uninitialized object of type NSWindow
[zoom_and_follow_mouse.py]   w = AppKit.NSWindow.alloc().initWithContentRect_styleMask_backing_defer_(frame, mask, AppKit.NSBackingStoreBuffered, False)
[zoom_and_follow_mouse.py] Traceback (most recent call last):
[zoom_and_follow_mouse.py]   File "/Users/ibby/Documents/OBS/scripts/obs-zoom-and-follow-master/zoom_and_follow_mouse.py", line 820, in toggle_zoom
[zoom_and_follow_mouse.py]     zoom.update_source_size()
[zoom_and_follow_mouse.py]   File "/Users/ibby/Documents/OBS/scripts/obs-zoom-and-follow-master/zoom_and_follow_mouse.py", line 283, in update_source_size
[zoom_and_follow_mouse.py]     self.update_window_dim(self.window)
[zoom_and_follow_mouse.py]   File "/Users/ibby/Documents/OBS/scripts/obs-zoom-and-follow-master/zoom_and_follow_mouse.py", line 69, in update_window_dim
[zoom_and_follow_mouse.py]     window_dim = window.getClientFrame()
[zoom_and_follow_mouse.py]   File "/usr/local/Cellar/python@3.10/3.10.7/Frameworks/Python.framework/Versions/Current/lib/python3.10/site-packages/pywinctl/_pywinctl_macos.py", line 490, in getClientFrame
[zoom_and_follow_mouse.py]     titleHeight, borderWidth = _getBorderSizes()
[zoom_and_follow_mouse.py]   File "/usr/local/Cellar/python@3.10/3.10.7/Frameworks/Python.framework/Versions/Current/lib/python3.10/site-packages/pywinctl/_pywinctl_macos.py", line 427, in _getBorderSizes
[zoom_and_follow_mouse.py]     w = AppKit.NSWindow.alloc().initWithContentRect_styleMask_backing_defer_(frame, mask, AppKit.NSBackingStoreBuffered, False)
[zoom_and_follow_mouse.py] objc.error: NSInternalInconsistencyException - NSWindow drag regions should only be invalidated on the Main Thread!
[zoom_and_follow_mouse.py] Tracking: False

I'm using a Homebrew installation of Python 3.10, and pywinctl version 0.0.38.

Also worth mentioning, I ran both test scripts here from Terminal. The test scripts prompted me to allow GUI scripting access via System Preferences.

I ticked the box next to Terminal in the screenshot, below, and was able to re-run the test scripts.

The test_pywinctl.py script failed on line 429. The test_MacNSWindow.py script was successful.

I'm wondering whether similar accessibility settings may be hindering pywinctl from working properly.

[Kalmat]

Hi! Thank you so much for your very detailed and careful description!

I think the obs-zoom crash is not related to accessibility options, because it's not used in this case. I think the problem is here:

 w = AppKit.NSWindow.alloc().initWithContentRect_styleMask_backing_defer_(frame, mask, AppKit.NSBackingStoreBuffered, False)
[zoom_and_follow_mouse.py] objc.error: NSInternalInconsistencyException - NSWindow drag regions should only be invalidated on the Main Thread!

What means it's been called in a separate thread, not main thread, but macOS won't allow that. I have answered your issue on obs-zoom site to try to find a solution together with its author (tryptech).

Apart from this, yes, it's totally necessary to grant accessibility permissions when using pywinctl to control other apps different from your own app. This is because macOS doesn't "like" to control windows from other apps, so the only way to do that is using AppleScript, which requires those permissions, apart from being tricky and slow... but there is no other choice, unfortunately.

What it's weird to me is the error you are pointing on line 429 of test_pywinctl.py... I will look into it!

Thank you again!

[Kalmat]

Hi again!

silly mistake in test_pywinctl.py. I must have pushed a test. The problem was that you have to allow some time for the window manager to refresh the changes. Just changing: timelap = 0.00 to timelap = 0.50 in basic_macOS, it works!

Thanks again!

[super-ibby]

Thank you, @Kalmat !

[super-ibby]

As mentioned in this obs_zoom_and_follow issue, I will open a PR in this repo to fix the issue.

---

Edit:

Silly me. I don't have permissions to branch on this repo (of course!). Please can you swap in the following for the _getBorderSizes() function in src/pywinctl/_pywinctl_macos.py?

def _getBorderSizes() -> tuple[int, int]:
    cmd = r"""
    use framework "Foundation"

    on run
        set theResult to {0, 0}
        tell (current application)
            my performSelectorOnMainThread:"getTitleBarHeightAndBorderWidth" withObject:0 waitUntilDone:true
        end tell
        return theResult
    end run

    on getTitleBarHeightAndBorderWidth()

        set |⌘| to current application

        set theFrameWidth to 250

        tell (|⌘|'s NSWindow's alloc()'s ¬
            initWithContentRect:{{400, 800}, {theFrameWidth, 100}} ¬
                styleMask:(|⌘|'s NSTitledWindowMask) ¬
                backing:(|⌘|'s NSBackingStoreBuffered) ¬
                defer:true)
            set theWindow to it
            set its releasedWhenClosed to true
            set its excludedFromWindowsMenu to true
        end tell

        set theFrame to theWindow's frame()

        set theTitleHeight to theWindow's titlebarHeight() as integer

        set theContentRect to theWindow's contentRectForFrameRect:theFrame

        set x1 to (|⌘|'s NSMinX(theContentRect)) as integer
        set x2 to (|⌘|'s NSMaxX(theContentRect)) as integer

        set theBorderWidth to (theFrameWidth - (x2 - x1)) as integer

        set my theResult to {theTitleHeight, theBorderWidth}

        theWindow's |close|()
        return
    end getTitleBarHeightAndBorderWidth"""
    proc = subprocess.Popen(
        ['osascript'],
        stdin=subprocess.PIPE,
        stdout=subprocess.PIPE,
        stderr=subprocess.PIPE,
        encoding='utf8',
    )
    ret, _ = proc.communicate(cmd)
    values = ret.replace("\n", "").strip().split(", ")
    return int(values[0]), int(values[1])

Thank you!

[Kalmat]

Hi! Thank you so much!!!

I can either swap your code or you can open a PR so I can approve it! As you prefer. In the meantime I will test this code in several macOS versions, which is usually problemmatic.

Besides, if you have AppleScript knowledge, and you have the time, I will really appreciate if you could help to improve what it is written in the module since, basically, I picked it from here and there, so I'm sure it's definitely improvable. Of course, no obligation, no rush, no worries. Than you again!

[Kalmat]

Hi again!

I made some quick tests using your script version. Please, find below the results:

High Sierra: Fails. I'm dumping here the error received. Any idea?

2023-02-02 09:22:25.060 osascript[467:5518] - [NSWindow titlebarHeight]: unrecognized selector sent to instance 0x7fd7203660a0
2023-02-02 09:22:25.060 osascript[467:5518] *** [BAGenericObjectNoDeleteOSAID getTitleBarHeightAndBorderWidth]: - [NSWindow titlebarHeight]: unrecognized selector sent to instance 0x7fd7203660a0 (error -10000)

Catalina: OK (title: 22, border: 0)

Monterrey: Couldn't test, my installation is "broken"

BigSur: OK (title: 28, border: 0)

Thanks!

[super-ibby]

Hi @Kalmat ,

I will open a PR.

Regarding the failure on High Sierra, could you try adding this to the beginning of the AppleScript in _getBorderSizes():

use AppleScript version "2.4" -- Yosemite (10.10) or later

Thank you!

[Kalmat]

Hi again!

Tested on High Sierra, now it doesn't fail, but it doesn't work (I mean, it does not return the proper titlebarHeight and borderSize values). I guess it was the expected behavior, right? Any idea about how to get the same values on High Sierra?

Thank you so much!

[super-ibby]

Hi! Unfortunately, I do not have a device running anything but BigSur. And I am extremely new to AppleScript.

Are you using a virtual machine to test code on different macOS versions? I could try that I suppose, but would take me some time to setup.

[Kalmat]

Nah, do not even worry. You've helped a lot so far. I will try to find a way. Thank you!

PS: to be "new" in AppleScript, yours is quite impressive!! HJAHAHAHA!

[super-ibby]

Hahaha thank you, took me hours of looking through stackoverflow and what not!

[super-ibby]

Fixed in https://github.com/Kalmat/PyWinCtl/commit/1d568c0ce09620ccbb7f7ebaeb7769620b1b66a7.