search

lbryio / lbry-sdk

The LBRY SDK for building decentralized, censorship resistant, monetized, digital content apps.
https://lbry.com
MIT License
7.2k stars 483 forks source link

Can't install libtorrent==2.0.7 dependency on MacOS (aarch64) #3669

Open moodyjon opened 1 year ago

moodyjon commented 1 year ago

This poses a problem for making the transition to native aarch64 python3.9. I don't know why it doesn't try to get libtorrent sources and compile in this case.

uname -a
Darwin Jonathans-Mac-mini.local 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:28:35 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T8101 arm64

(lbry-venv-3.9) swdev1@Jonathans-Mac-mini lbry-sdk % pip install -e .
Obtaining file:///Users/swdev1/lbry-sdk
  Preparing metadata (setup.py) ... done
Requirement already satisfied: aiohttp==3.7.4 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (3.7.4)
Requirement already satisfied: aioupnp==0.0.18 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (0.0.18)
Requirement already satisfied: appdirs==1.4.3 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (1.4.3)
Requirement already satisfied: certifi>=2021.10.08 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (2022.5.18.1)
Requirement already satisfied: colorama==0.3.7 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (0.3.7)
Requirement already satisfied: distro==1.4.0 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (1.4.0)
Requirement already satisfied: base58==1.0.0 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (1.0.0)
Requirement already satisfied: cffi==1.13.2 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (1.13.2)
Collecting cryptography==2.5
  Using cached cryptography-2.5.tar.gz (487 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: protobuf==3.17.2 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (3.17.2)
Requirement already satisfied: prometheus_client==0.7.1 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (0.7.1)
Requirement already satisfied: ecdsa==0.13.3 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (0.13.3)
Requirement already satisfied: pyyaml==5.3.1 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (5.3.1)
Requirement already satisfied: docopt==0.6.2 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (0.6.2)
Requirement already satisfied: hachoir==3.1.2 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (3.1.2)
Requirement already satisfied: coincurve==15.0.0 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (15.0.0)
Requirement already satisfied: pbkdf2==1.3 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (1.3)
Requirement already satisfied: filetype==1.0.9 in ./lbry-venv-3.9/lib/python3.9/site-packages (from lbry==0.110.0) (1.0.9)
ERROR: Could not find a version that satisfies the requirement libtorrent==2.0.7 (from lbry) (from versions: none)
ERROR: No matching distribution found for libtorrent==2.0.7
WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/Users/swdev1/lbry-sdk/lbry-venv-3.9/bin/python3.9 -m pip install --upgrade pip' command.

I have an old libtorrent installed from the time I was using python3.7 x86_64 architecture (emulated). I think I was getting away with running tests before because libtorrent was not required to run the tests I wanted to run. But it's required after https://github.com/lbryio/lbry-sdk/commit/5be990fc55fd717f1bb29b5ae54dfcb6f0fe48e3

(lbry-venv-3.9) swdev1@Jonathans-Mac-mini lbry-sdk % env ENABLE_LEGACY_SEARCH=1 python -m unittest -vv -k test_single_server_payment tests/integration/blockchain/test_wallet_server_sessions.py
test_wallet_server_sessions (unittest.loader._FailedTest) ... ERROR

======================================================================
ERROR: test_wallet_server_sessions (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: test_wallet_server_sessions
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.13_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/Users/swdev1/lbry-sdk/tests/integration/blockchain/test_wallet_server_sessions.py", line 9, in <module>
    from lbry.testcase import IntegrationTestCase, CommandTestCase
  File "/Users/swdev1/lbry-sdk/lbry/testcase.py", line 25, in <module>
    from lbry.extras.daemon.daemon import Daemon, jsonrpc_dumps_pretty
  File "/Users/swdev1/lbry-sdk/lbry/extras/daemon/daemon.py", line 46, in <module>
    from lbry.extras.daemon.components import WALLET_COMPONENT, DATABASE_COMPONENT, DHT_COMPONENT, BLOB_COMPONENT
  File "/Users/swdev1/lbry-sdk/lbry/extras/daemon/components.py", line 31, in <module>
    from lbry.torrent.session import TorrentSession
  File "/Users/swdev1/lbry-sdk/lbry/torrent/session.py", line 10, in <module>
    import libtorrent
ImportError: dlopen(/Users/swdev1/lbry-sdk/lbry-venv-3.9/lib/python3.9/site-packages/libtorrent.so, 0x0002): tried: '/Users/swdev1/lbry-sdk/lbry-venv-3.9/lib/python3.9/site-packages/libtorrent.so' (not a mach-o file)
moodyjon commented 1 year ago

Currently working around this with local edits:

(lbry-venv-3.9) swdev1@Jonathans-Mac-mini lbry-sdk % git diff
diff --git a/lbry/extras/daemon/components.py b/lbry/extras/daemon/components.py
index 33deccee9..0a005b068 100644
--- a/lbry/extras/daemon/components.py
+++ b/lbry/extras/daemon/components.py
@@ -28,7 +28,10 @@ from lbry.torrent.torrent_manager import TorrentManager
 from lbry.wallet import WalletManager
 from lbry.wallet.usage_payment import WalletServerPayer
 from lbry.torrent.tracker import TrackerClient
-from lbry.torrent.session import TorrentSession
+try:
+    from lbry.torrent.session import TorrentSession
+except ImportError:
+    TorrentSession = None

 log = logging.getLogger(__name__)
robd003 commented 1 year ago

Opened a ticket so hopefully libtorrent will release builds for macOS aarch64: https://github.com/arvidn/libtorrent/issues/7131

moodyjon commented 1 year ago

I have made some progress on this. I was able to get libtorrent to build from source and install, but only in a Python 3.9 environment. I had the same experience working on https://github.com/arvidn/libtorrent/issues/7131 itself. Could not build MacOS arm64 wheels for Python 3.8. Only wheels for 3.9 and 3.10.

robd003 commented 1 year ago

Shouldn't the goal be upgrading LBRY to the most recent version of Python? 3.9 only receives security fixes. 3.10 still gets bug fixes.

I'm pretty sure Python 3.8 isn't even compatible with macOS 11+