Kalmat / PyWinCtl

Cross-Platform module to get info on and control windows on screen
Other
193 stars 20 forks source link

Apple Silicon MacOS getActiveWindowTitle() large latency #84

Open Sky-Unmodal opened 8 months ago

Sky-Unmodal commented 8 months ago

Following is the code I used in my case:

import pywinctl as pwc
import sys
import time
from pynput import mouse

def on_click(x, y, button, pressed):
     start = time.time()
     if pressed:
         print(f"Mouse button {button} pressed at position ({x}, {y})")
         try:
             print(pwc.getActiveWindow().title)
             print("Latency:",time.time() - start)
         except AttributeError:
             print("No active window")
     else:
         print(f"Mouse button {button} released at position ({x}, {y})")

 with mouse.Listener(on_click=on_click) as listener:
        listener.join()

I ran this on a M1 Macbook pro. The latency is around 400ms to 500ms to just get the active window.

I ran this on another windows machine, the same code gave a latency around 1ms to almost none latency.

May I ask why does the performance vary this much? Or is there something I should do in MacOS for it run smoother?

Kalmat commented 8 months ago

Hi! As you can read in the README page... (HAHAHAHA! Nobody does, I know, including myself!)

***Important macOS notice ***

macOS doesn't "like" controlling windows from other apps, so there are two separate classes you can use:
- To control your own application's windows: MacOSNSWindow() is based on NSWindow Objects (you have to pass the NSApp() and the NSWindow() objects reference).
- To control other applications' windows: MacOSWindow() is based on Apple Script, so it is non-standard, slower and, in some cases, tricky (uses window name as reference, which may change or be duplicate), but it's working fine in most cases. You will likely need to grant permissions on Settings -> Security&Privacy -> Accessibility. ***Notice some applications will have limited Apple Script support or no support at all, so some or even all methods may fail!***

This is really annoying, but this is all we can do when on macOS, sorry for the bad news. The difference in performance is because, on macOS, you have to open a terminal, load the script interpreter, run the script, return the values... 400-500ms as result.

I hope you can manage to find a workaround. Just let me know!

Sky-Unmodal commented 8 months ago

Hi Kalmat! Thanks for the timely response. I realized that the getActiveWindowTitle() function for macos is using Applescript and I actually tried running a less complex Applescript to get the app's title in Python. The latency for that is still too high for me, around 250ms.

Later, I found an alternative to get the application name (not the window name but the app's name) using AppKit.

def get_active_application_macos():
    from AppKit import NSWorkspace
    active_app = NSWorkspace.sharedWorkspace().frontmostApplication().localizedName()
    return active_app

With this, I can get the the current application's name within 1ms latency on a Macbook M1 pro. I am not sure about whether we can control that app using AppKit as my goal is just getting the current active app on macos.

Thanks!

Kalmat commented 8 months ago

Hi again! Thank you for your help!

I forgot to mention that pywinctl.getActiveWindow().title actually runs two scripts (one to get the active window, and the other to get its title). It can be improved by using pywinctl.getActiveWindowTitle() which only runs one script, but still too slow...

In addition to that, PyWinCtl is intended to get info and control windows, not apps. You can use getApp() method, but it is just the name of the parent app, not an object you can use in any other processes.

As far as I could find, the only way to manipulate other apps' windows in macOS is using AppleScript. The AppKit app object will not let you change any property. Besides, getting the title of the app can be enough for mono-window apps, but not for multi-window apps (e.g. TextEdit will return 'TextEdit' when invoking app.localizedName(), but the active window title will be something like "TextEdit - " + "the-name-of-the-doc-you-are-editing").

In your case, if you just need the name of the active app, you can use the snippet you attached, which is way faster!! Just bear in mind that for non-English languages the localizedName() might be different (e.g. in Spanish localizedName() method for the app "Contacts" will return "Contactos")

Thank you again for your help and interest! Just let me know any thought or improvement you can find!!!

ocn commented 2 months ago

Hi there, sorry for bumping an old thread, but is there anyway to get individual window titles from multi-window apps? I am looking to find multiple windows of the same bundle and get their window titles independently.

Kalmat commented 2 months ago

Hi there! Sorry for my late reply. I have limited access to my development stuff these days.

Don't worry about bumping, though it is better for other users to open separate posts for separate issues.

In your case, using pywinctl.getAllAppsWindowsTitles() should do the trick. Check this and please let me know if this works for you (I am not sure this will work for all applications):

import pywinctl as pwc

print("WINDOWS")
print(pwc.getAllTitles())
print("APPS")
print(pwc.getAllAppsNames())
print("APPS AND WINDOWS")
print(pwc.getAllAppsWindowsTitles())

In a very simple scenario, the output is as follows:

WINDOWS
['Proyectos', 'tests — osascript ◂ Python test.py — 80×24', 'Untitled 2', 'Untitled', 'National and Local Weather Radar, Daily Forecast, Hurricane and information from The Weather Channel and weather.com']
APPS
['Finder', 'Terminal', 'TextEdit', 'Safari']
APPS AND WINDOWS
{'Finder': ['Proyectos'], 'Terminal': ['tests — osascript ◂ Python test.py — 80×24'], 'TextEdit': ['Untitled 2', 'Untitled'], 'Safari': ['National and Local Weather Radar, Daily Forecast, Hurricane and information from The Weather Channel and weather.com']}

Note that TextEdit returns two windows (I already opened two empty docs in separate windows). whilst Safari only returns the active tab (there were several actually open, but just one visible).

Hope this helps!

ocn commented 3 weeks ago

Howdy, thanks for responding and sorry for my late response.

So, fun fact, it seems that if a macOS app/window is full screened (in my case, two instances of a particular app), there's no window title:

>>> pprint(pwc.getAllAppsWindowsTitles())
...
 'exefile': [],

however if they are both windowed, there is in fact a window title:

 'exefile': ['EVE - Draconic Slayer'],

This is quite inconvenient for my use-case. I'm trying to setup a program where I can setup hotkeys to swap to a particular fullscreened window/app/whatever, however there's multiple instances of exefile that would have different window titles (to use as a key) were they not fullscreen.

Separately, and this may warrant a new issue, it seems to only update the windows I have open once-per-restart. This doesn't sound like an issue with pywinctl, though. I just opened a few more instances of my app (exefile), but they did not populate in the above function call (when repeated).

Kalmat commented 2 weeks ago

Hi!

In my case, the behaviour is the opposite: when a window is maximized, then it is the only one found when using getAllTitles() or getAllAppsWindowsTitles() (also when trying to find other windows using getAllwindows()).

Testing this simple script in Big Sur (sorry, I have no access to a more modern version):

import time

import pywinctl as pwc

print("INIT")
for w in pwc.getAllWindows():
    print("getAllWindows", w.title)
print("getAllTitles", pwc.getAllTitles())
print("getAllAppsWindowsTitles", pwc.getAllAppsWindowsTitles())
print()

win = pwc.getWindowsWithTitle("Sin ", condition=pwc.Re.STARTSWITH)[0]

while True:
    try:
        if win.isMaximized:
            print("WHILE MAXIMIZED")
            for w in pwc.getAllWindows():
                print("getAllWindows", w.title)
            print("getAllTitles", pwc.getAllTitles())
            print("getAllAppsWindowsTitles", pwc.getAllAppsWindowsTitles())
            print()
        time.sleep(1)
    except KeyboardInterrupt:
        break

The output is (I maximized the "TextEdit - Sin título" window while the script was running):

INIT
getAllWindows tests — osascript ◂ Python test2.py — 80×24
getAllWindows Sin título
getAllTitles ['tests — osascript ◂ Python test2.py — 80×24', 'Sin título']
getAllAppsWindowsTitles {'Finder': [], 'Terminal': ['tests — osascript ◂ Python test2.py — 80×24'], 'TextEdit': ['Sin título']}

WHILE MAXIMIZED
getAllWindows Sin título
getAllTitles ['Sin título']
getAllAppsWindowsTitles {'Finder': [], 'Terminal': [], 'TextEdit': ['Sin título']}

This is can be very inconvenient in some use cases... I have to figure out how to fix it. I suspect the key is in this condition I use within AppleScript to avoid showing non-user (system) processes: of (every process whose background only is false), which turns out to take a value of true for all windows except for the one which is maximized.

I don't know if this all helps you in any way, but according to your explanation, you should be able to find the maximized window (and only that one). If this not the case, please let me know so I can try to help you. In the meantime, I will try to figure out the "good" solution... if any.