Closed janit closed 3 years ago
I'm pretty sure that only Python 3.9 is aware of the M1 ARM issues. Can you try it on 3.9 and report back?
I upgraded to Python 3.9 via Brew and the issue persists.
See log with version data:
administrator@mx1 chia-blockchain % python3 --version
Python 3.9.2
administrator@mx1 chia-blockchain % pip3 --version
pip 21.0.1 from /opt/homebrew/lib/python3.9/site-packages/pip (python 3.9)
administrator@mx1 chia-blockchain % sh install.sh
Python version is 3.9
Requirement already satisfied: pip in ./venv/lib/python3.9/site-packages (21.0.1)
Requirement already satisfied: wheel in ./venv/lib/python3.9/site-packages (0.36.2)
Looking in indexes: https://pypi.org/simple, https://download.chia.net/simple/
Requirement already satisfied: miniupnpc==2.1 in ./venv/lib/python3.9/site-packages (2.1)
Looking in indexes: https://pypi.org/simple, https://download.chia.net/simple/
Obtaining file:///Users/administrator/chia-blockchain
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting aiosqlite@ git+https://github.com/mariano54/aiosqlite.git@47c7b21dd04adb1d41073ee9911a9d4b9c4b370f#egg=aiosqlite
Cloning https://github.com/mariano54/aiosqlite.git (to revision 47c7b21dd04adb1d41073ee9911a9d4b9c4b370f) to /private/var/folders/_0/yxj9brhd1g1174_kk_cpjdd80000gn/T/pip-install-l7u1s79i/aiosqlite_73223847fab04ea3861d72f0ebd3f824
Running command git clone -q https://github.com/mariano54/aiosqlite.git /private/var/folders/_0/yxj9brhd1g1174_kk_cpjdd80000gn/T/pip-install-l7u1s79i/aiosqlite_73223847fab04ea3861d72f0ebd3f824
Running command git rev-parse -q --verify 'sha^47c7b21dd04adb1d41073ee9911a9d4b9c4b370f'
Running command git fetch -q https://github.com/mariano54/aiosqlite.git 47c7b21dd04adb1d41073ee9911a9d4b9c4b370f
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing wheel metadata ... done
Collecting clvm==0.8.9
Downloading clvm-0.8.9-py3-none-any.whl (18 kB)
Collecting clvm-tools==0.3.5
Using cached clvm_tools-0.3.5-py3-none-any.whl (35 kB)
Collecting setproctitle==1.2.2
Using cached setproctitle-1.2.2.tar.gz (23 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting websockets==8.1.0
Using cached websockets-8.1.tar.gz (58 kB)
Collecting keyring==21.5.0
Downloading keyring-21.5.0-py3-none-any.whl (32 kB)
Collecting concurrent-log-handler==0.9.19
Downloading concurrent_log_handler-0.9.19-py2.py3-none-any.whl (18 kB)
Collecting chiavdf==0.15.0
Using cached chiavdf-0.15.0-cp39-cp39-macosx_10_14_universal2.whl (316 kB)
Collecting keyrings.cryptfile==1.3.4
Downloading keyrings.cryptfile-1.3.4.tar.gz (17 kB)
Collecting aiter==0.13.20191203
Using cached aiter-0.13.20191203.tar.gz (9.3 kB)
Collecting cryptography==3.3.2
Using cached cryptography-3.3.2.tar.gz (539 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting bitstring==3.1.7
Using cached bitstring-3.1.7.tar.gz (195 kB)
Collecting sortedcontainers==2.3.0
Downloading sortedcontainers-2.3.0-py2.py3-none-any.whl (29 kB)
Collecting chiapos==0.12.44
Downloading chiapos-0.12.44-cp39-cp39-macosx_10_14_universal2.whl (215 kB)
|████████████████████████████████| 215 kB 638 kB/s
Collecting blspy==0.3.5
Using cached blspy-0.3.5-cp39-cp39-macosx_10_14_universal2.whl (642 kB)
Collecting chiabip158==0.19
Downloading chiabip158-0.19-cp39-cp39-macosx_10_14_universal2.whl (277 kB)
|████████████████████████████████| 277 kB 3.7 MB/s
Collecting PyYAML==5.4.1
Downloading PyYAML-5.4.1.tar.gz (175 kB)
|████████████████████████████████| 175 kB 4.4 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
ERROR: Could not find a version that satisfies the requirement clvm-rs==0.1.3 (from chia-blockchain)
ERROR: No matching distribution found for clvm-rs==0.1.3
administrator@mx1 chia-blockchain %
Thanks for trying 3.9 as our M1 is stuck in a reboot loop but this is a good reason to fix that. I'll dig into this shortly.
Thanks. For anyone hit with this issue, I did manage to plot on an M1 Mac with the Parallels preview release with Ubuntu 20.04. No problems there.
I'm pretty sure I know what's going on but it's going to require an M1 CI runner. I ordered one overnight so should have a solution in the next couple of days.
Great. Interesting to see how this performs natively on macOS vs the virtualized Ubuntu.
I have hand rolled clvm-rs==0.1.3 and put it on our pypi simple clone. RC2 just built fine for me on M1 and so did pre-release RC3.
I will be automating building both clvm-rs and an Apple Silicon native installer shortly.
Once you confirm it builds for you I will close this.
Issues building the GUI though.
I can confirm it builds now! 👍
But note that this is only after an upgrade to Python 3.9 from Brew. So maybe add that into either the installation instructions or then to the installer itself. For those struggling with this issue in the meanwhile, just do the upgrade manually:
$ brew install python@3.9
Hold that. The build goes through, but there's an issue when running chia init
:
Traceback (most recent call last):
File "/Users/administrator/chia-blockchain/venv/bin/chia", line 33, in <module>
sys.exit(load_entry_point('chia-blockchain', 'console_scripts', 'chia')())
File "/Users/administrator/chia-blockchain/src/cmds/chia.py", line 57, in main
parser = create_parser()
File "/Users/administrator/chia-blockchain/src/cmds/chia.py", line 45, in create_parser
mod = importlib.import_module("src.cmds.%s" % subcommand)
File "/opt/homebrew/Cellar/python@3.9/3.9.2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 790, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/Users/administrator/chia-blockchain/src/cmds/init.py", line 9, in <module>
from src.util.keychain import Keychain
File "/Users/administrator/chia-blockchain/src/util/keychain.py", line 12, in <module>
from blspy import AugSchemeMPL, G1Element, PrivateKey
ImportError: dlopen(/Users/administrator/chia-blockchain/venv/lib/python3.9/site-packages/blspy.cpython-39-darwin.so, 2): no suitable image found. Did find:
/Users/administrator/chia-blockchain/venv/lib/python3.9/site-packages/blspy.cpython-39-darwin.so: mach-o, but wrong architecture
/Users/administrator/chia-blockchain/venv/lib/python3.9/site-packages/blspy.cpython-39-darwin.so: mach-o, but wrong architecture
Not much of a Python guy, but looks like it could do with some mismatch of the two installed versions (default and from Brew). From what I understand the virtual environment should handle this, but maybe it picks the wrong one.
Did init
work on your end?
I tested it a different way so didn't see the bls issue. I'll chase it in the AM. Could be a problem using the universal builds instead of just building for intel and m1.
Ahha... Could be. Also didn't have Rosetta installed, since I reset the whole shebang. I'll try adding that and see if it runs through that automagically. Ideally should be all native, obviously, but to get it running would be another step forward.
The universal builds we're doing on python 3.9 should run natively on arm64 but clearly it's not getting packaged right. I'll switch to just making an intel and an arm64 version now that I know we can create an arm64/M1 only installer
Assuming the installers build correctly - here is one more issue dealt with: https://github.com/Chia-Network/chia-blockchain/pull/1062
Great, but I suppose this won't help with the issue with init
on CLI?
Update - first the bad news. The cross compile method we were hoping would work doesn't due to our reliance on things like gmp and libsodium. Also, we can't get our M1 into the CI flow to automate building these.
Better news - it looks like we can get our M1 able to build chia-blockchain and all of the dependencies natively - though by hand. I'll pull a dmg together in a few days but it requires a lot of small changes.
I've successfully installed on M1 (but beware of https://github.com/Chia-Network/chia-blockchain/issues/1189).
Right now RC5 will build from sh install.sh
and I should be able to keep publishing the various dependencies by hand at the releases until the world gets M1 in a CI somewhere.
I can confirm the build, activation and plotting now run on my M1 Mac Mini environment. Thanks for your efforts!
The only snag was that the installer does not install some needed brew packages, leading to the following errors:
File "/Users/administrator/chia-blockchain/src/util/keychain.py", line 12, in <module>
from blspy import AugSchemeMPL, G1Element, PrivateKey
ImportError: dlopen(/Users/administrator/chia-blockchain/venv/lib/python3.9/site-packages/blspy.cpython-39-darwin.so, 2): Library not loaded: /opt/homebrew/opt/libsodium/lib/libsodium.23.dylib
Referenced from: /Users/administrator/chia-blockchain/venv/lib/python3.9/site-packages/blspy.cpython-39-darwin.so
Reason: image not found
and
File "/Users/administrator/chia-blockchain/src/util/keychain.py", line 12, in <module>
from blspy import AugSchemeMPL, G1Element, PrivateKey
ImportError: dlopen(/Users/administrator/chia-blockchain/venv/lib/python3.9/site-packages/blspy.cpython-39-darwin.so, 2): Library not loaded: /opt/homebrew/opt/gmp/lib/libgmp.10.dylib
Referenced from: /Users/administrator/chia-blockchain/venv/lib/python3.9/site-packages/blspy.cpython-39-darwin.so
Reason: image not found
Installing both packages manually fixed the issue:
$ brew install libsodium gmp
Totally forgot to statically link those as that gets done automatically on CI. I'll replace that wheel tonight so you don't need the dependencies.
Thanks!
I fixed both chiavdf and blspy. You should be able to brew uninstall libsodium gmp
and then sh install.sh
happily.
Removed libsodium and gmp. Then installed, activated and started a plot without issues. Perfect.
On an M1 (ARM64) I cannot build the development version because of this error in install:
This is very similar to the issue #980 with the same error on Raspberry Pi, where it was caused by a 32 bit installation. I tried a few older betas, another version of Python3 from Brew, to no avail.
How to reproduce on a Mac with M1 ARM CPU:
Here is a full log of the installer until the error: