Volume control fix for MacOS

shubham1172 commented 3 years ago

Fixes #5

This PR adds an extra step to the package workflow that runs the python script fix_pyautogui_macos.py. We need to do this to fix https://github.com/asweigart/pyautogui/issues/37. The script hotfixes the code changes that need to be done in pyautogui and we are hoping to clean this up as soon as the original issue is resolved.

Also, in main.py, press special volume keys if it's MacOS.

Signed-off-by: Shubham Sharma shubhash@microsoft.com

shubham1172 commented 3 years ago

@jjcrab, can you please test this on macOS?

In a python interpreter, get the path to the file that we will change.

fpath = os.path.join(os.path.dirname(
    pyautogui.__file__), "_pyautogui_osx.py")

Copy the original file into another location to take a backup (Very important)
Run the script python hack/fix_pyautogui_macos.py
Start the web server and check if the volume buttons work!

jjcrab commented 3 years ago

I am not sure how do I run hack/fix_pyautogui_macos.py since the changes are all in your branch. Can you create a dev branch and push these changes there so I can checkout to it and give it a try? Or please suggest anything else you have in mind. Thanks!

shubham1172 commented 3 years ago

https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/syncing-a-fork You can sync your fork with this repo and get my branch. After this, you should be able to git checkout shubham1172/volume-control-fix-bug-#5 and run the file.

Alternatively, if it's a hassle, you can clone this repo directly and checkout shubham1172/volume-control-fix-bug-#5.

jjcrab commented 3 years ago

Thanks for the suggestion. I added your fork upstream and was able to checkout to your branch. I backed up my original _pyautogui_osx.py file and ran your hack/fix_pyautogui_macos.py script.

I then tried to restart the web server and running in to the error as shown below:

$ uvicorn main:app --reload --host 0.0.0.0
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [86991] using statreload
/opt/homebrew/lib/python3.9/site-packages/rubicon/objc/ctypes_patch.py:21: UserWarning: rubicon.objc.ctypes_patch has only been tested with Python 3.4 through 3.8. You are using Python 3.9.4. Most likely things will work properly, but you may experience crashes if Python's internals have changed significantly.
  warnings.warn(
Process SpawnProcess-1:
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/homebrew/lib/python3.9/site-packages/uvicorn/subprocess.py", line 61, in subprocess_started
    target(sockets=sockets)
  File "/opt/homebrew/lib/python3.9/site-packages/uvicorn/server.py", line 49, in run
    loop.run_until_complete(self.serve(sockets=sockets))
  File "/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
  File "/opt/homebrew/lib/python3.9/site-packages/uvicorn/server.py", line 56, in serve
    config.load()
  File "/opt/homebrew/lib/python3.9/site-packages/uvicorn/config.py", line 308, in load
    self.loaded_app = import_from_string(self.app)
  File "/opt/homebrew/lib/python3.9/site-packages/uvicorn/importer.py", line 20, in import_from_string
    module = importlib.import_module(module_str)
  File "/opt/homebrew/Cellar/python@3.9/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "./main.py", line 2, in <module>
    import pyautogui
  File "/opt/homebrew/lib/python3.9/site-packages/pyautogui/__init__.py", line 536, in <module>
    from . import _pyautogui_osx as platformModule
  File "/opt/homebrew/lib/python3.9/site-packages/pyautogui/_pyautogui_osx.py", line 434
    n == RIGHT:
               ^
SyntaxError: invalid syntax

shubham1172 commented 3 years ago

If you open the file _pyautogui_osx.py in a text editor, do you see any syntax errors? I personally don't see any on my PC and not sure why this won't work.

One possibility can be that the standard python interfaces work slightly differently on macOS.

Can you also dump your python and OS version here?

shubham1172 commented 3 years ago

@jjcrab any luck testing it?

jjcrab commented 3 years ago

hi @shubham1172 sorry for the delay. I have been working on a project which has a deadline on next Monday, so I haven't got a chance to check this yet. But I will work on this project after next Monday.

shubham1172 commented 3 years ago

Hey, no worries. I was just curious to know if this would at all work or not. If not, I will need to have a closer look at the PyAutoGui source code!!

jjcrab commented 3 years ago

I looked at the line it was complaining about and an "if" was missing from there. I added it and it compiled successfully.

$ uvicorn main:app --reload --host 0.0.0.0 INFO: Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit) INFO: Started reloader process [10185] using statreload /opt/homebrew/lib/python3.9/site-packages/rubicon/objc/ctypes_patch.py:21: UserWarning: rubicon.objc.ctypes_patch has only been tested with Python 3.4 through 3.8. You are using Python 3.9.4. Most likely things will work properly, but you may experience crashes if Python's internals have changed significantly. warnings.warn( INFO: Started server process [10187] INFO: Waiting for application startup. INFO: Application startup complete. INFO: 192.168.0.27:55876 - "GET / HTTP/1.1" 200 OK INFO: 192.168.0.27:55876 - "GET /static/style.css HTTP/1.1" 404 Not Found INFO: 192.168.0.27:55877 - "GET /static/index.js HTTP/1.1" 404 Not Found INFO: 192.168.0.27:55878 - "GET /favicon.ico HTTP/1.1" 200 OK

But now when I none of the controls are working and it is giving 404 errors as shown above

jjcrab commented 3 years ago

Update:

I commented out this section of the code from your main.py and now I can connect the app and the volume controls are working too along with all the other controls.

commented out the below:

# if __name__ == "__main__":

#     # See https://stackoverflow.com/a/42615559/4654175
#     if getattr(sys, 'frozen', False):
#         # If the application is run as a bundle, the PyInstaller bootloader
#         # extends the sys module by a flag frozen=True and sets the app
#         # path into variable _MEIPASS'.
#         # Since _MEIPASS does not exists at development time,
#         # pylint needs to be suppressed.
#         application_path = sys._MEIPASS  # pylint: disable=no-member
#     else:
#         application_path = os.path.dirname(os.path.abspath(__file__))

#     app.mount(
#         "/static", StaticFiles(directory=os.path.join(application_path, "static")))

#     host = os.environ.get("REMOTE_PLAY_HOST", "0.0.0.0")
#     port = os.environ.get("REMOTE_PLAY_PORT", 8000)

#     uvicorn.run(app, host=host, port=port)

shubham1172 commented 3 years ago

@jjcrab oh no, I will add a Contributing.MD file #18 to fix this confusion. The instructions have changed, instead of running uvicorn manually, you can simply run python main.py and you won't need to comment anything out.

I looked at the line it was complaining about and an "if" was missing from there.

Where was it? Can you add the snippet around that here? AFAIK line 434 is towards the end of the file that we didn't even change.

jjcrab commented 3 years ago

here is the snippet. the if was missing before n == RIGHT I am curious too how this got modified if you did not change it via your fix script.

def _moveTo(x, y):
    _sendMouseEvent(Quartz.kCGEventMouseMoved, x, y, 0)
    time.sleep(0.01)  # needed to allow OS time to catch up.
    if n == RIGHT:
        _sendMouseEvent(Quartz.kCGEventRightMouseDragged,
                        x, y, Quartz.kCGMouseButtonRight)
    else:
        assert False, "button argument not in ('left', 'middle', 'right')"
    time.sleep(0.01)  # needed to allow OS time to catch up.

shubham1172 commented 3 years ago

Can you restore your backup and try again? I don't think this should be the problem, because it's not doing that on my computer.

At this point I am also in favor of just closing this PR (since after adding the if it works on your machine too) and then you can download the mac executable and test directly.

jjcrab commented 3 years ago

Sorry, I have been a bit busy with some personal projects. I think we can close this and follow your recommendation.

shubham1172 / remote-play

Volume control fix for MacOS #14