Kalmat / PyWinCtl

Cross-Platform module to get info on and control windows on screen
Other
179 stars 19 forks source link

Windows 10 + WSL2 Not working #71

Closed mhl787156 closed 1 year ago

mhl787156 commented 1 year ago

Hi! I'm currently building an application out of various python TK/QT/WX Guis (Don't ask...requirements...) and do a bunch of my dev on WSL2 on windows. PyWinCTL looks great for my application, but it's not quite working on WSL in particular I think.

I have a basic script:

import subprocess
import pywinctl as pwc
import time
command = ["python", "fleet_monitor.py", "example/basic_tee_spatially_separated_mission", "-n", "5"]
subprocess.Popen(command)
time.sleep(5)
titles = pwc.getAllTitles()
print(titles)
print(pwc.getAllWindows())
print(pwc.getAllAppsWindowsTitles())

which opens a TKinter window and the tries to find it. At which point it fails to find these windows and all the calls to pwc return empty.

I then cloned the repo and ran pytest tests/test_pywinctl.py - where it FAILED tests/test_pywinctl.py::test_basic - KeyError: 'XDG_CURRENT_DESKTOP'. Not sure what the correct entry is on WSL... but I set export XDG_CURRENT_DESKTOP=ubuntu:GNOME and that seemed to mollify it.

Running the tests again I now get an error about Xlib which I'm not sure how to resolve which is below.

=================================================================== test session starts ====================================================================
platform linux -- Python 3.9.16, pytest-7.4.0, pluggy-1.2.0
rootdir: /home/mickey/Portal/PyWinCtl
collected 1 item

tests/test_pywinctl.py F                                                                                                                             [100%]

========================================================================= FAILURES =========================================================================
________________________________________________________________________ test_basic ________________________________________________________________________

    def test_basic():

        if sys.platform == "win32":
            subprocess.Popen('notepad')
            time.sleep(0.5)

            testWindows = [pywinctl.getActiveWindow()]
            # testWindows = pywinctl.getWindowsWithTitle('Untitled - Notepad')   # Not working in other languages
            assert len(testWindows) == 1

            npw = testWindows[0]

            basic_win32(npw)

        elif sys.platform == "linux":
            subprocess.Popen('gedit')
            time.sleep(5)

            testWindows = [pywinctl.getActiveWindow()]
            assert len(testWindows) == 1

            npw = testWindows[0]

>           basic_linux(npw)

tests/test_pywinctl.py:37:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tests/test_pywinctl.py:245: in basic_linux
    if npw.isMaximized:  # Make sure it starts un-maximized
../../mambaforge/envs/portal/lib/python3.9/site-packages/pywinctl/_pywinctl_linux.py:746: in isMaximized
    state = self._win.getWmState(True)
../../mambaforge/envs/portal/lib/python3.9/site-packages/ewmhlib/_ewmhlib.py:1324: in getWmState
    return getPropertyValue(self.getProperty(Window.WM_STATE), text, self.display)
../../mambaforge/envs/portal/lib/python3.9/site-packages/ewmhlib/_ewmhlib.py:1032: in getProperty
    return getProperty(self.xWindow, prop, prop_type, self.display)
../../mambaforge/envs/portal/lib/python3.9/site-packages/ewmhlib/_ewmhlib.py:180: in getProperty
    return window.get_full_property(prop, prop_type)
../../mambaforge/envs/portal/lib/python3.9/site-packages/Xlib/xobject/drawable.py:472: in get_full_property
    prop = self.get_property(property, property_type, 0, sizehint)
../../mambaforge/envs/portal/lib/python3.9/site-packages/Xlib/xobject/drawable.py:455: in get_property
    r = request.GetProperty(display = self.display,
../../mambaforge/envs/portal/lib/python3.9/site-packages/Xlib/protocol/rq.py:1368: in __init__
    self.reply()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <GetProperty serial = 19, data = None, error = <class 'Xlib.error.BadWindow'>: code = 3, resource_id = <Resource 0x00c00115>, sequence_number = 19, major_opcode = 20, minor_opcode = 0>

    def reply(self):
        # Send request and wait for reply if we hasn't
        # already got one.  This means that reply() can safely
        # be called more than one time.

        self._response_lock.acquire()
        while self._data is None and self._error is None:
            self._display.send_recv_lock.acquire()
            self._response_lock.release()

            self._display.send_and_recv(request = self._serial)
            self._response_lock.acquire()

        self._response_lock.release()
        self._display = None

        # If error has been set, raise it
        if self._error:
>           raise self._error
E           Xlib.error.BadWindow: <class 'Xlib.error.BadWindow'>: code = 3, resource_id = <Resource 0x00c00115>, sequence_number = 19, major_opcode = 20, minor_opcode = 0

../../mambaforge/envs/portal/lib/python3.9/site-packages/Xlib/protocol/rq.py:1388: BadWindow
================================================================= short test summary info ==================================================================
FAILED tests/test_pywinctl.py::test_basic - Xlib.error.BadWindow: <class 'Xlib.error.BadWindow'>: code = 3, resource_id = <Resource 0x00c00115>, sequence_number = 19, major_opcode = 20, minor_opc...
==================================================================== 1 failed in 5.34s =====================================================================

I'm assuming there's some libraries that WSL2 is missing or oddly configured by default which PyWinCtl might need?

Thanks!

Kalmat commented 1 year ago

Wow! Interesting project and very challenging question!

PyWinCtl relies heavily on X11 and EWMH. Since WSL2 doesn't have any X-Server, it will not work. You would need to install a X-Server for Windows which, in addtion, is compliant with EWMH specs (don't even know if that exists). Even though you manage to do this, I am not sure it will fully work (I never tested myself, to be honest).

I am not familiar with WSL2, unfortunately. In a quick search, I found this X-Server for Windows, which seems to have EWMH support thru XCB, as per this, but just guessing.

Not sure if that all fits your needs or make any sense at all... Please let me know what you think.

Thank you!

mhl787156 commented 1 year ago

Hi!

Haha I was worried that WSL might have this sort of problem! I might have a go at installing the X-Server but it might be a bit too much of a pain to get it to work nicely. It's weird because I assume that there is some sort of X-server running as gedit and TKinter pop up windows when natively running in WSL2 - I think it might have changed recently but I can't find anything online (You used to need Vcxsrv and set DISPLAY as shown in the medium article but now dont)

Update - This Microsoft Link shows that it now runs X11 and Wayland(?) by default?

Update #2 - The github might be useful understanding its architecture: https://github.com/microsoft/wslg#wslg-architecture-overview

My idea was to use some sort of window manager to manage the running and layout of all of the different windows so they fit onto one screen - inspired in part by how you can define executables and layouts in tmux. Are you aware of any other tools I might be able to try? I've read about i3, but dont know if its right for me.

Kalmat commented 1 year ago

Hi again!

Just guessing: did you check if those windows are included in pywinctl.getAllWindows() when running it on Windows itself? I don't really think so but, if they do, you can manage them from Windows. If it works, you can take a look to this which, if I understood you well, it is very similar to what you are looking for.

Sticking to WSL2/Linux, apart from X11, it is necessary EWMH. For instance, Wayland is not totally compliant with EWMH, so there is no way to get the active window nor the list of open windows (making it impossible to address just one window unless you already have its XID)... This drove me crazy for weeks until I eventually had to desist. According to the architecture you linked, it seems to be using Wayland as compositor. If so, you can give a try to this:

import json
import subprocess
from typing import List, Union

cmd = ('gdbus call --session --dest org.gnome.Shell --object-path /org/gnome/Shell '
       '--method org.gnome.Shell.Eval "global.get_window_actors()'
       '.map(a=>a.meta_window)'
       '.map(w=>({class: w.get_wm_class(), title: w.get_title(), active: w.has_focus(), id: w.get_description(), id2: w.get_id(), id3: w.get_pid()}))"')
ret = subprocess.check_output(cmd, shell=True, timeout=1).decode("utf-8").replace("\n", "")
if ret and ret.startswith("(true, "):
    windows: List[str] = (str(ret[8:-2]).replace("[", "").replace("]", "").replace("},{", "}|&|{").split("|&|"))
    for window in windows:
        output: dict[str, Union[str, bool]] = json.loads(window)
        print(output)
else:
    print(ret)

Check if the target windows are printed using this script. If it complains about permissions, you will likely have to enable unsafe-mode... If so, it will not be usable for end-users (this is why I had to desist), but it can give you a good reference on what may be happening and how to address it.

Unfortunately, I am not aware of those tools you mention. I am quite new in all this about programming, and I code just as a hobby, sorry, so there are lots of tools and languages and environments I am not familiar with!

... A mess, I know... but, who said it was going to be easy? HAHAHAHA!

Kalmat commented 1 year ago

EDIT I managed to install WSL2 on my system (just curious). Running pywinctl.getAllWindows() from Windows, it finds the Ubuntu window, and it can be targeted. Not sure if everything will properly work yet, but I was able to get and move the window from Windows itself.

Captura

Within WSL (Ubuntu), the gdbuscommand fails (most likely because of the unsafe-mode not enabled, but not sure).

Apart from this, do you have to stick to WSL? Or you can give a try to a VM on, e.g., Virtualbox, or even better install Ubuntu along with Windows (it can be done without affecting your whole system nor the boot procedure)?

mhl787156 commented 1 year ago

Thanks for trying it out, yes I tried it over the weekend and the gdbus command fails for me too. I guess WSL passes off the creation and management of the window to the Windows systems.

I could definitely do that, but they want a kinda cross-platform solution without VMs and such. We have one application which only works on Windows, and another only on Linux - WSL conveniently manages to connect them together is all! I might just move over to Linux proper and ditch the windows application and call it a day, or do some funky TKinter things to deliver what they want.

Anyhow thanks for looking into it!

Kalmat commented 1 year ago

One last thing, just in case it is helpful: If you just want to control your own application windows, you can do so by instantiating pywinctl.Window(xid). You can get the xid value for your target window by using PyQt's winId() or Tkinter's root.frame() methods...

Thank YOU for your feedback!