start-quic-tunnel fails to run (missing: homebrew openssl package)

WaterJuice commented 1 year ago

Test environment

MacOS 13.4.1. Python 3.11.1
iPhone 14 Pro. iOS 17.0 Beta 3
pymobiledevice3 2.3.1

Describe the bug Running the command sudo python3 -m pymobiledevice3 remote start-quic-tunnel causes an error NameError: name 'create_core_device_tunnel_service' is not defined

Logs

% sudo python3 -m pymobiledevice3 remote start-quic-tunnel
Readline features including tab completion have been disabled because
no supported version of readline was found. To resolve this, install
pyreadline3 on Windows or gnureadline on Linux/Mac.

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pymobiledevice3/__main__.py", line 102, in <module>
    cli()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pymobiledevice3/__main__.py", line 61, in cli
    cli_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pymobiledevice3/cli/remote.py", line 98, in cli_start_quic_tunnel
    asyncio.run(start_quic_tunnel(rsd, secrets), debug=True)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pymobiledevice3/cli/remote.py", line 67, in start_quic_tunnel
    with create_core_device_tunnel_service(service_provider, autopair=True) as service:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
NameError: name 'create_core_device_tunnel_service' is not defined

doronz88 commented 1 year ago

Looks like from some reason you got an ImportError, which is expected to happen only on Windows. Can you test what import failed?

WaterJuice commented 1 year ago

Looks like from some reason you got an ImportError, which is expected to happen only on Windows. Can you test what import failed?

How?

doronz88 commented 1 year ago

Try the import yourself:

from pymobiledevice3.remote.core_device_tunnel_service import create_core_device_tunnel_service

WaterJuice commented 1 year ago

Try the import yourself:

from pymobiledevice3.remote.core_device_tunnel_service import create_core_device_tunnel_service


% python3
Python 3.11.1 (v3.11.1:a7a450f84a, Dec  6 2022, 15:24:06) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pymobiledevice3.remote.core_device_tunnel_service import create_core_device_tunnel_service
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/pymobiledevice3/remote/core_device_tunnel_service.py", line 19, in <module>
    from aioquic_pmd3.asyncio import QuicConnectionProtocol
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/asyncio/__init__.py", line 1, in <module>
    from .client import connect  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/asyncio/client.py", line 8, in <module>
    from ..quic.connection import QuicConnection
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/quic/connection.py", line 20, in <module>
    from .crypto import CryptoError, CryptoPair, KeyUnavailableError
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/quic/crypto.py", line 4, in <module>
    from .._crypto import AEAD, CryptoError, HeaderProtection
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/_crypto.abi3.so, 0x0002): Library not loaded: /opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib
  Referenced from: <1BCD490E-70E3-3B52-863B-3C59B5119BA9> /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/_crypto.abi3.so
  Reason: tried: '/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib' (no such file), '/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib' (no such file), '/usr/lib/libcrypto.3.dylib' (no such file, not in dyld cache)

doronz88 commented 1 year ago

Looks like there's something wrong with your openssl installation. Either way I'll add a commit to not silently ignore this error.

doronz88 commented 1 year ago

I think this should fix it:

brew install openssl@3

WaterJuice commented 1 year ago

% which openssl
/usr/bin/openssl
% openssl version
LibreSSL 3.3.6

I don't have brew. I'd rather not install that as it messes up a bunch of things. Is there a way to use pymobiledevice3 with the existing openssl installed?

doronz88 commented 1 year ago

I don't have another solution for that. That is required for the QUIC implementation. The dependency is as follows:

pymobiledevice3 -> aioquic-pmd3 (fork of aioquic) -> homebrew's openssl@3

If you can create a PR that fixes this dependency for aioquic-pmd3 I'll be happy to merge it

doronz88 commented 1 year ago

@WaterJuice can you verify #551 fixes the dependency issue?

WaterJuice commented 1 year ago

@WaterJuice can you verify #551 fixes the dependency issue?

% sudo python3 -m pymobiledevice3 remote start-quic-tunnel
Readline features including tab completion have been disabled because
no supported version of readline was found. To resolve this, install
pyreadline3 on Windows or gnureadline on Linux/Mac.

2023-09-03 15:42:50 carrots-MacBook-Pro.local pymobiledevice3.cli.remote[86982] WARNING create_core_device_tunnel_service failed to be imported. Some feature may not work.
You can debug this by trying the import yourself:

from pymobiledevice3.remote.core_device_tunnel_service import create_core_device_tunnel_service
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/carrot/Downloads/pymobiledevice3-refactor-aioquic_pmd3_pure_python/pymobiledevice3/__main__.py", line 104, in <module>
    cli()
  File "/Users/carrot/Downloads/pymobiledevice3-refactor-aioquic_pmd3_pure_python/pymobiledevice3/__main__.py", line 61, in cli
    cli_commands()
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/carrot/Downloads/pymobiledevice3-refactor-aioquic_pmd3_pure_python/pymobiledevice3/cli/remote.py", line 122, in cli_start_quic_tunnel
    asyncio.run(start_quic_tunnel(rsd, secrets), debug=True)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/carrot/Downloads/pymobiledevice3-refactor-aioquic_pmd3_pure_python/pymobiledevice3/cli/remote.py", line 70, in start_quic_tunnel
    with create_core_device_tunnel_service(service_provider, autopair=True) as service:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
NameError: name 'create_core_device_tunnel_service' is not defined

Looks the same to me.

doronz88 commented 1 year ago

1 - Did you re-install this version? 2 - Can you test the import suggested?

WaterJuice commented 1 year ago

1 - Did you re-install this version? 2 - Can you test the import suggested?

This is running from a directory I just downloaded from that branch

% python3
Python 3.11.1 (v3.11.1:a7a450f84a, Dec  6 2022, 15:24:06) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from pymobiledevice3.remote.core_device_tunnel_service import create_core_device_tunnel_service
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/carrot/Downloads/pymobiledevice3-refactor-aioquic_pmd3_pure_python/pymobiledevice3/remote/core_device_tunnel_service.py", line 19, in <module>
    from aioquic_pmd3.asyncio import QuicConnectionProtocol
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/asyncio/__init__.py", line 1, in <module>
    from .client import connect  # noqa
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/asyncio/client.py", line 8, in <module>
    from ..quic.connection import QuicConnection
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/quic/connection.py", line 20, in <module>
    from .crypto import CryptoError, CryptoPair, KeyUnavailableError
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/quic/crypto.py", line 4, in <module>
    from .._crypto import AEAD, CryptoError, HeaderProtection
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/_crypto.abi3.so, 0x0002): Library not loaded: /opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib
  Referenced from: <1BCD490E-70E3-3B52-863B-3C59B5119BA9> /Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/aioquic_pmd3/_crypto.abi3.so
  Reason: tried: '/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib' (no such file), '/opt/homebrew/opt/openssl@3/lib/libcrypto.3.dylib' (no such file), '/usr/lib/libcrypto.3.dylib' (no such file, not in dyld cache)

doronz88 commented 1 year ago

This doesn't answer my question. Did you re-install the package?
Also, please print the aioquic-pmd3 version. It should be 0.0.2 which removed the openssl requirement

WaterJuice commented 1 year ago

Oh I see what happened. So I didn't pip install pymobiledevice3 this time, I just downloaded the .zip and ran it from the directory. However I had aioquic-pmd3 installed from when I did the pip install previously. So I upgraded that and now it works. Thanks!

% sudo python3 -m pymobiledevice3 remote start-quic-tunnel
Readline features including tab completion have been disabled because
no supported version of readline was found. To resolve this, install
pyreadline3 on Windows or gnureadline on Linux/Mac.

UDID: 00008120-0002596A0198C01E
ProductType: iPhone15,2
ProductVersion: 17.0
Interface: utun8
RSD Address: fdbf:a849:75f9::1
RSD Port: 57204
Use the follow connection option:
--rsd fdbf:a849:75f9::1 57204

doronz88 commented 1 year ago

Cool! 🥳

doronz88 / pymobiledevice3

start-quic-tunnel fails to run (missing: homebrew openssl package) #527