Closed OvoNemaSmisla closed 3 years ago
Something aarch64 specific? Has somebody tried running JM on 64-bit ARM before?
Anyone?
Seems to be the same issue as https://github.com/kristapsk/raspibolt-extras/issues/5.
Guess the problem is that nobody from JM devs have aarch64 hardware to try to reproduce...
That's unfortunate :(
Is this related? https://github.com/ofek/coincurve/issues/71
You should maybe focus on how to get coincurve to install (it would just be pip install coincurve
), it looks from the log that that's the only issue.
I have a plan to factor out the remaining dependencies on coincurve as with a little work we should be able to replace the remaining dependencies on it, with python-bitcointx
(which we are now using for most of the secp256k1 stuff), but it's a non-trivial bit of work and I don't have time right now. I'm not sure why coincurve isn't installing now, I think it must have done in the past.
Successfully installed coincurve with pip install coincurve
.
The installation still fails with following errors:
Building wheels for collected packages: twisted, coincurve, pyaes, urldecode, argon2-cffi, bencoder.pyx, zope.interface, future
Building wheel for twisted (setup.py) ... done
Created wheel for twisted: filename=Twisted-20.3.0-cp36-cp36m-linux_aarch64.whl size=3078836 sha256=96682f06a7d41aff0abf188e49306ec1e9b0364511d7316146298e6ebdb601bf
Stored in directory: /home/user/.cache/pip/wheels/45/9b/0f/0a3b9d4efa96ecc139fb6fa5a172b11ea9f745895236e36017
Building wheel for coincurve (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/user/joinmarket-clientserver/jmvenv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8ks46u9a/coincurve/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8ks46u9a/coincurve/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-7pbzny_j
cwd: /tmp/pip-install-8ks46u9a/coincurve/
Complete output (25 lines):
running bdist_wheel
The [wheel] section is deprecated. Use [bdist_wheel] instead.
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.6
creating build/lib.linux-aarch64-3.6/coincurve
copying coincurve/__init__.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/_windows_libsecp256k1.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/context.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/keys.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/utils.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/ecdsa.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/flags.py -> build/lib.linux-aarch64-3.6/coincurve
running build_ext
generating cffi module 'build/temp.linux-aarch64-3.6/_libsecp256k1.c'
creating build/temp.linux-aarch64-3.6
building '_libsecp256k1' extension
creating build/temp.linux-aarch64-3.6/build
creating build/temp.linux-aarch64-3.6/build/temp.linux-aarch64-3.6
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -I/home/user/joinmarket-clientserver/jmvenv/include/python3.6m -c build/temp.linux-aarch64-3.6/_libsecp256k1.c -o build/temp.linux-aarch64-3.6/build/temp.linux-aarch64-3.6/_libsecp256k1.o
aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-aarch64-3.6/build/temp.linux-aarch64-3.6/_libsecp256k1.o -lsecp256k1 -o build/lib.linux-aarch64-3.6/coincurve/_libsecp256k1.cpython-36m-aarch64-linux-gnu.so
/usr/bin/ld: cannot find -lsecp256k1
collect2: error: ld returned 1 exit status
error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for coincurve
Running setup.py clean for coincurve
Building wheel for pyaes (setup.py) ... done
Created wheel for pyaes: filename=pyaes-1.6.1-py3-none-any.whl size=26346 sha256=dad067226cd10caa29fed67d6f7bbed7f2aed2d13c0ab4a198495696e302e029
Stored in directory: /home/user/.cache/pip/wheels/0b/7a/53/f1d7837e24ac5d8c0d06b7cbe2144be66dc5e7f37e02f5a164
Building wheel for urldecode (setup.py) ... done
Created wheel for urldecode: filename=urldecode-0.1-py3-none-any.whl size=1543 sha256=e0dc253142c76e4085e0be12ece200a92f91593214112aba8ccee1fddeb7b552
Stored in directory: /home/user/.cache/pip/wheels/9b/52/b5/34da44490b0c78379205e8743253bc3c358e48ea84f9a40988
Building wheel for argon2-cffi (PEP 517) ... done
Created wheel for argon2-cffi: filename=argon2_cffi-20.1.0-cp36-abi3-linux_aarch64.whl size=83460 sha256=84abec0cefa98e64502f026f2daa5a0176ab6460d331bc6b7ae5c784d4027b28
Stored in directory: /home/user/.cache/pip/wheels/b0/d3/08/2934319e5d942ed093aa126b91eb5eab77c5a49af021dd7f22
Building wheel for bencoder.pyx (setup.py) ... done
Created wheel for bencoder.pyx: filename=bencoder.pyx-2.0.1-cp36-cp36m-linux_aarch64.whl size=169488 sha256=7308091051e8b347e03b815f4502a39c745bbeb30e656c7666445600660a0165
Stored in directory: /home/user/.cache/pip/wheels/72/3b/7b/1e45fb0f21872df32e1ec62f39ac5a22c489625d49a1e99c5f
Building wheel for zope.interface (setup.py) ... done
Created wheel for zope.interface: filename=zope.interface-5.1.0-cp36-cp36m-linux_aarch64.whl size=222624 sha256=b146632decca96b5228410152b31a8497f6b0801d8efe7f9b6c3943b9319e0e1
Stored in directory: /home/user/.cache/pip/wheels/4d/b1/d4/071ce5e5e22295b5c70b72f4df166ea45f8a60c9374c9feb3c
Building wheel for future (setup.py) ... done
Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491059 sha256=94ca35a0c6d927539cc5d52948333cb591ad0c7701be1df13f99b39075ea1549
Stored in directory: /home/user/.cache/pip/wheels/6e/9c/ed/4499c9865ac1002697793e0ae05ba6be33553d098f3347fb94
Successfully built twisted pyaes urldecode argon2-cffi bencoder.pyx zope.interface future
Failed to build coincurve
Installing collected packages: zope.interface, constantly, incremental, attrs, six, Automat, idna, hyperlink, PyHamcrest, twisted, pycparser, cffi, cryptography, pyasn1, pyasn1-modules, service-identity, future, colorama, chromalog, joinmarketbase, asn1crypto, coincurve, python-bitcointx, pyaes, urldecode, joinmarketbitcoin, mnemonic, argon2-cffi, bencoder.pyx, joinmarketclient, txtorcon, pyopenssl, libnacl, joinmarketdaemon
Running setup.py develop for joinmarketbase
Running setup.py install for coincurve ... error
ERROR: Command errored out with exit status 1:
command: /home/user/joinmarket-clientserver/jmvenv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8ks46u9a/coincurve/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8ks46u9a/coincurve/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-kvhe7oy3/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/joinmarket-clientserver/jmvenv/include/site/python3.6/coincurve
cwd: /tmp/pip-install-8ks46u9a/coincurve/
Complete output (24 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-aarch64-3.6
creating build/lib.linux-aarch64-3.6/coincurve
copying coincurve/__init__.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/_windows_libsecp256k1.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/context.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/keys.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/utils.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/ecdsa.py -> build/lib.linux-aarch64-3.6/coincurve
copying coincurve/flags.py -> build/lib.linux-aarch64-3.6/coincurve
running build_ext
generating cffi module 'build/temp.linux-aarch64-3.6/_libsecp256k1.c'
creating build/temp.linux-aarch64-3.6
building '_libsecp256k1' extension
creating build/temp.linux-aarch64-3.6/build
creating build/temp.linux-aarch64-3.6/build/temp.linux-aarch64-3.6
aarch64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -I/home/user/joinmarket-clientserver/jmvenv/include/python3.6m -c build/temp.linux-aarch64-3.6/_libsecp256k1.c -o build/temp.linux-aarch64-3.6/build/temp.linux-aarch64-3.6/_libsecp256k1.o
aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-aarch64-3.6/build/temp.linux-aarch64-3.6/_libsecp256k1.o -lsecp256k1 -o build/lib.linux-aarch64-3.6/coincurve/_libsecp256k1.cpython-36m-aarch64-linux-gnu.so
/usr/bin/ld: cannot find -lsecp256k1
collect2: error: ld returned 1 exit status
error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /home/user/joinmarket-clientserver/jmvenv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-8ks46u9a/coincurve/setup.py'"'"'; __file__='"'"'/tmp/pip-install-8ks46u9a/coincurve/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-kvhe7oy3/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/joinmarket-clientserver/jmvenv/include/site/python3.6/coincurve Check the logs for full command output.
Joinmarket was not installed. Exiting.
As far as I can see it starts with building wheels for a few packages:
Building wheels for collected packages: twisted, coincurve, pyaes, urldecode, argon2-cffi, bencoder.pyx, zope.interface, future
It manages to create the wheels for all except coincurve:
Building wheel for coincurve (setup.py) ... error ... ERROR: Failed building wheel for coincurve
Then it tries to install all the packages and crashes when it gets to coincurve:
Running setup.py install for coincurve ... error ... /usr/bin/ld: cannot find -lsecp256k1 collect2: error: ld returned 1 exit status error: command 'aarch64-linux-gnu-gcc' failed with exit status 1
(edited to remove unrelated stuff)
I'm having a similar issue when attempting to install on FreeBSD. Same as above, when building building coincurve it fails with the following output:
Collecting coincurve
Using cached coincurve-13.0.0.tar.gz (961 kB)
Requirement already satisfied: asn1crypto in ./jmvenv/lib/python3.7/site-packages (from coincurve) (1.4.0)
Requirement already satisfied: cffi>=1.3.0 in ./jmvenv/lib/python3.7/site-packages (from coincurve) (1.14.3)
Requirement already satisfied: pycparser in ./jmvenv/lib/python3.7/site-packages (from cffi>=1.3.0->coincurve) (2.20)
Building wheels for collected packages: coincurve
Building wheel for coincurve (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/joinmarket/jmvenv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x2qaf5y4/coincurve/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x2qaf5y4/coincurve/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-wz3evbic
cwd: /tmp/pip-install-x2qaf5y4/coincurve/
Complete output (27 lines):
Warning: 'keywords' should be a list, got type 'tuple'
running bdist_wheel
The [wheel] section is deprecated. Use [bdist_wheel] instead.
running build
running build_py
creating build
creating build/lib.freebsd-12.1-RELEASE-amd64-3.7
creating build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/flags.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/context.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/_windows_libsecp256k1.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/ecdsa.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/utils.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/keys.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/__init__.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
running build_ext
generating cffi module 'build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.c'
creating build/temp.freebsd-12.1-RELEASE-amd64-3.7
building '_libsecp256k1' extension
creating build/temp.freebsd-12.1-RELEASE-amd64-3.7/build
creating build/temp.freebsd-12.1-RELEASE-amd64-3.7/build/temp.freebsd-12.1-RELEASE-amd64-3.7
cc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -fPIC -I/usr/local/include/python3.7m -c build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.c -o build/temp.freebsd-12.1-RELEASE-amd64-3.7/build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.o
build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.c:569:10: fatal error: 'secp256k1.h' file not found
#include <secp256k1.h>
^~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for coincurve
Running setup.py clean for coincurve
Failed to build coincurve
Installing collected packages: coincurve
Running setup.py install for coincurve ... error
ERROR: Command errored out with exit status 1:
command: /usr/local/bin/joinmarket/jmvenv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x2qaf5y4/coincurve/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x2qaf5y4/coincurve/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-39c_fjnb/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/bin/joinmarket/jmvenv/include/site/python3.7/coincurve
cwd: /tmp/pip-install-x2qaf5y4/coincurve/
Complete output (26 lines):
Warning: 'keywords' should be a list, got type 'tuple'
running install
running build
running build_py
creating build
creating build/lib.freebsd-12.1-RELEASE-amd64-3.7
creating build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/flags.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/context.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/_windows_libsecp256k1.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/ecdsa.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/utils.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/keys.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
copying coincurve/__init__.py -> build/lib.freebsd-12.1-RELEASE-amd64-3.7/coincurve
running build_ext
generating cffi module 'build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.c'
creating build/temp.freebsd-12.1-RELEASE-amd64-3.7
building '_libsecp256k1' extension
creating build/temp.freebsd-12.1-RELEASE-amd64-3.7/build
creating build/temp.freebsd-12.1-RELEASE-amd64-3.7/build/temp.freebsd-12.1-RELEASE-amd64-3.7
cc -pthread -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -fPIC -I/usr/local/include/python3.7m -c build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.c -o build/temp.freebsd-12.1-RELEASE-amd64-3.7/build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.o
build/temp.freebsd-12.1-RELEASE-amd64-3.7/_libsecp256k1.c:569:10: fatal error: 'secp256k1.h' file not found
#include <secp256k1.h>
^~~~~~~~~~~~~
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/joinmarket/jmvenv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-x2qaf5y4/coincurve/setup.py'"'"'; __file__='"'"'/tmp/pip-install-x2qaf5y4/coincurve/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-39c_fjnb/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/bin/joinmarket/jmvenv/include/site/python3.7/coincurve Check the logs for full command output.
So not exactly the same, but very close:
fatal error: 'secp256k1.h' file not found
@bektar, I tested in July with FreeBSD 12.1 and in theory all you should do now before running install.sh
is pkg install bash python py37-virtualenv autotools gmake gcc pkgconf py-sqlite3 py37-sqlite3
, see #630.
@kristapsk that works mostly, but when following the steps found under the heading "Alternative/custom installation" on https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/docs/INSTALL.md it fails.
i.e. the commands
/usr/local/bin/bash -c "source $INTERNAL_BIN_PATH/jmvenv/bin/activate; python setupall.py --daemon"
/usr/local/bin/bash -c "source $INTERNAL_BIN_PATH/jmvenv/bin/activate; python setupall.py --client-bitcoin"
and specifically the "python setupall.py --client-bitcoin" one
Ehm, and please ignore the extra cruft around it, it is basically to make my scripts prettier and reusable. It doesn't affect anything pertinent to this (i.e. if i manually enter the venv and run the commands the result is the same)
@bektar First, yes secp256k1-py
is no longer relevant, we replaced it with coincurve
, second I think since you're installing on a different system then please open a separate issue to discuss that and remove the above, because it'll make this Issue very confusing for readers (especially with the secp stuff which isn't relevant to either issue).
@OvoNemaSmisla I haven't commented further since your last update because I don't know what's actually happening here.
I was hoping that someone like @openoms , i.e. more specifically someone who does install Joinmarket on RPi (I don't/haven't) would chime in, since clearly the process was working correctly at some point (perhaps still is? it seems many people have done it), and I'm not aware of what have might changed. In particular on pip install coincurve
, that is exactly what should happen as a result of this line:
https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmbitcoin/setup.py#L13
... so I can't understand why it isn't working during install if it works manually otherwise.
One idea occurs to me, depending on the distro, the default python, and pip, may be python2 not 3, that is just a low-probability guess as to why the pip installation of coincurve works outside of the install (the install is all python3 now).
Come to think of it, that is the biggest/most important difference between 0.7.0 Joinmarket and the previous versions - we've switched to requiring python3.6+ (see release notes) and cannot use py3.5 any more. Does 0.6.3.1 install OK?
Yeah, it started to feel a bit off topic. But there is some connection due to it appears as if coincurve fails to build due to
…
aarch64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-aarch64-3.6/build/temp.linux-aarch64-3.6/_libsecp256k1.o -lsecp256k1 -o build/lib.linux-aarch64-3.6/coincurve/_libsecp256k1.cpython-36m-aarch64-linux-gnu.so
/usr/bin/ld: cannot find -lsecp256k1
…
Anyway, I'll open a different issue and reference this one from there
Hmm, perhaps I was wrong then. But maybe just remove the secp stuff.
I'll edit my posts… just need to figure out in what way. Right now when I'm running python setupall.py --client-bitcoin again I actually got no errors. Going to reset the system and start over and try to figure out what the hell is going on. I'm sure it is my fault somehow, just now how.
@bektar , why do you want to use python setupall.py
, not install.sh
?
@kristapsk I'll move all the purely freebsd related stuff to #630
Not sure if this is of any help to the actual topic, but might be. I suspect the error can be solved on freebsd by installing py37-secp256k1 and/or py37-coincurve, and then opening a new shell. After installing them it still didn't work, but opening a new bash and running "source jmvenv/bin/activate" again I can now run "python setupall.py --client-bitcoin" without errors. (Still figuring out the specifics)
No idea if there are any similar packages to be found for aarch64 though.
@waxwing on the RPi (with the most popular Debian based Raspberry OS) the environment is 32bit armv7l
using armhf
packages.
This Odroid board is running a 64 bit aarch64
OS which is less tested, but should be supported.
Regarding the problematic package you can try to install the package with apt
or find an alternative one for pip
, install it globally and use --system-site-packages
with the jmvenv
. Would need to import that specific package every time when activating the virtual environment.
Example of this is here to replace the missing pyQT5
dependencies for armhf
:
https://github.com/openoms/joininbox/blob/master/scripts/install.joinmarket.sh#L15
@openoms are you able to install 0.7.0 OK?
Another quick update from my end, I've managed to run "python setupall.py --client-bitcoin" without errors at least twice, but I cannot figure out how to reproduce it. My conjecture above doesn't seem to be correct, and that is just about as much as I can conclude atm.
What makes this much more frustrating is I'm basically running everything from a script and then resetting the jail after each try, but it still sometimes works and sometimes doesn't. Anyway, I'm going to step away from the computer now for a while and will try to get back to you later once I have more conclusive data.
Off topic: Is there any irc channel or discord or something similar where you guys communicate? It feels like most of what I'm writing turns out to be wrong, and I don't like editing or deleting posts since it makes the conversation so strange.
@AdamISZ 0.7.0 works ok with standard 32-bit RPi, I tested before bumping version in my raspibolt-extras JM guide, problem is only on aarch64 (64-bit ARM).
@kristapsk I'll move all the purely freebsd related stuff to #630
Better open new issue, that one is closed PR to add FreeBSD compatibility to install.sh
. It was tested both by me and @jules23 and it worked.
Is there any irc channel or discord or something similar where you guys communicate? It feels like most of what I'm writing turns out to be wrong
Yes, #joinmarket on freenode
@OvoNemaSmisla I haven't commented further since your last update because I don't know what's actually happening here.
I was hoping that someone like @openoms , i.e. more specifically someone who does install Joinmarket on RPi (I don't/haven't) would chime in, since clearly the process was working correctly at some point (perhaps still is? it seems many people have done it), and I'm not aware of what have might changed. In particular on
pip install coincurve
, that is exactly what should happen as a result of this line:https://github.com/JoinMarket-Org/joinmarket-clientserver/blob/master/jmbitcoin/setup.py#L13
... so I can't understand why it isn't working during install if it works manually otherwise.
One idea occurs to me, depending on the distro, the default python, and pip, may be python2 not 3, that is just a low-probability guess as to why the pip installation of coincurve works outside of the install (the install is all python3 now).
Come to think of it, that is the biggest/most important difference between 0.7.0 Joinmarket and the previous versions - we've switched to requiring python3.6+ (see release notes) and cannot use py3.5 any more. Does 0.6.3.1 install OK?
@AdamISZ I got 2 versions of pyton:
user@odroid:~$ python2 -V
Python 2.7.17
user@odroid:~$ python3 -V
Python 3.6.9
Default version was 2.7.17, so I've changed it with update-alternatives
to 3.6.9.
I'll try to pip install coincurve
to check if it is successful and will also try the version 0.6.3.1.
Regarding the problematic package you can try to install the package with
apt
or find an alternative one forpip
, install it globally and use--system-site-packages
with thejmvenv
. Would need to import that specific package every time when activating the virtual environment.
@openoms I'd like to try your suggestion also, but don't know which package are you referring to.
I've tried the install with default python version changed to 3.6.9 but no success. The same issue happens.
I've ran pip install coincurve
and it was successful again with following output:
pip install coincurve
Collecting coincurve
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/cf/d2/93372689e3054b8f0b8c 1946706c2d8f66a233b6200a2042f485b38871fd/coincurve-13.0.0.tar.gz (961kB)
100% |################################| 962kB 596kB/s
Collecting asn1crypto (from coincurve)
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/b5/a8/56be92dcd4a5bf199870 5a9b4028249fe7c9a035b955fe93b6a3e5b829f8/asn1crypto-1.4.0-py2.py3-none-any.whl ( 104kB)
100% |################################| 112kB 1.6MB/s
Collecting cffi>=1.3.0 (from coincurve)
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/cb/ae/380e33d621ae30177035 8eb11a896a34c34f30db188847a561e8e39ee866/cffi-1.14.3.tar.gz (470kB)
100% |################################| 471kB 1.1MB/s
Collecting pycparser (from cffi>=1.3.0->coincurve)
Cache entry deserialization failed, entry ignored
Cache entry deserialization failed, entry ignored
Downloading https://files.pythonhosted.org/packages/ae/e7/d9c3a176ca4b02024deb f82342dab36efadfc5776f9c8db077e8f6e71821/pycparser-2.20-py2.py3-none-any.whl (11 2kB)
100% |################################| 112kB 3.4MB/s
Building wheels for collected packages: coincurve, cffi
Running setup.py bdist_wheel for coincurve ... done
Stored in directory: /home/truly/.cache/pip/wheels/e5/6c/ed/d374aca4f0f08c17f15db41d54065b016b89a56d328525eb34
Running setup.py bdist_wheel for cffi ... done
Stored in directory: /home/truly/.cache/pip/wheels/61/02/c7/1d824371b744863db703d79708fc7f4c3bfa79da15b5aa9171
Successfully built coincurve cffi
Installing collected packages: asn1crypto, pycparser, cffi, coincurve
Successfully installed asn1crypto-1.4.0 cffi-1.14.3 coincurve-13.0.0 pycparser-2.20
The last thing I tried is to install 0.6.3.1 version and it was successful.
pip install coincurve
solves the above issue in some cases.
I have coincurve 13.0.0 installed for the user and the joinmarket install.sh
takes that the requiremenet is already satisfied.
Odroid C4 running 64bit Armbian Focal with Python 3.8.5:
$ uname -a
Linux odroidc4 5.8.5-meson64 #20.08.1 SMP PREEMPT Tue Sep 1 18:47:20 CEST 2020 aarch64 aarch64 aarch64 GNU/Linux
Odroid HC2 running 32bit Armbian Buster with an older kernel and Python 3.7.3:
$ uname -a
Linux odroidxu4 4.14.141-odroidxu4 #10 SMP PREEMPT Mon Sep 2 01:52:32 CEST 2019 armv7l GNU/Linux
Strange because the coincurve version is not pinned: https://github.com/JoinMarket-Org/joinmarket-clientserver=/blob/be117fca202b3edfd226143247f7ab3e7110ab3e/jmbitcoin/setup.py#L13 , so it is not clear why installing for the system makes a difference.
$ uname -a
Linux odroidc4 5.8.16-meson64 #20.08.15 SMP PREEMPT Sat Oct 24 20:12:56 CEST 2020 aarch64 GNU/Linux
needed to add in
sudo apt install libffi-dev
(from https://github.com/ofek/coincurve#installation)
For the record all the dependencies installed for coincurve
are:
sudo apt install build-essential automake pkg-config libtool libffi-dev python3-dev libgmp-dev
pip install asn1crypto cffi pycparser coincurve
Would worth to add all these to the installation to avoid obscure missing packages.
not related to systemwide or not, I just did . jmvenv/bin/activate pip install coincurve exit ./install.sh again and that worked for me (answer no to all questions)
libffi and headers are compiled and installed just becore coincurve in install.sh so you should not need the system package
@ketominer Sounds very strange and kinda makes no sense, coincurve is just dependency of jmbitcoin and should be installed by install.sh. @openoms reminded me that RPi3 has aarch64 capable board, maybe I should try to reproduce with a different SD card with 64-bit Linux and try figure out what's happening there.
yes definitely strange!
I was running 0.6.3.1 version for a few weeks without any problems.
After reading previous posts, I tried to upgrade to 0.7.2 version and it was successful.
I'm not sure if sudo -u joinmarket pip install coincurve
or adding the dependencies resolved the issue, but it's working now.
@kristapsk @ketominer do you see any issue with skipping to build libffi
from source if it is already present systemwide?
Running sudo apt install libffi-dev
before installing coincurve helped my aarch64 Focal build and libffi
is installed with apt
for example for Specter too.
@OvoNemaSmisla @openoms is there anything actionable here still, or should it be closed?
Looks like and edge case and unclear what the solution was here so can be closed. Will reopen if can reproduce it reliably.
@openoms So you are able to install JM on aarch64 Linux without any issues now? From my part the problem was that I didn't have aarch64 Linux to reproduce. There is related issue https://github.com/kristapsk/raspibolt-extras/issues/5.
As detailed above the install scripts in both raspiblitz
https://github.com/rootzoll/raspiblitz/blob/64ee445b251ad3f59609e5f3cbcb24cd0e7ce664/home.admin/config.scripts/bonus.joinmarket.sh#L104
and joininbox: https://github.com/openoms/joininbox/blob/7b0022ec368c9329ecc763d5ca85ac4e06261386/scripts/_functions.sh#L169
add a bunch of dependencies:
# PySide2 for armf: https://packages.debian.org/buster/python3-pyside2.qtcore
echo "# installing ARM specific dependencies to run the QT GUI"
sudo apt install -y python3-pyside2.qtcore python3-pyside2.qtgui \
python3-pyside2.qtwidgets zlib1g-dev libjpeg-dev python3-pyqt5 libltdl-dev
# https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/668#issuecomment-717815719
sudo apt -y install build-essential automake pkg-config libffi-dev python3-dev libgmp-dev
sudo -u joinmarket pip install libtool asn1crypto cffi pycparser coincurve
RaspiBlitz (and likely RaspiBolt too) will be based on the 64bit RaspberryOS (https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2020-08-24/) in the future: https://github.com/rootzoll/raspiblitz/issues/1199#issuecomment-736564381. Still deciding for the best base image for a JoininBox SDcard image, will likely test this on to be consistent with 64bit and use Debian with minimal bloat: https://raspi.debian.net/
For me, the 0.7.2 version worked. I'm not sure why, the system hasn't changed.
The reason might be the additional dependencies that 0.7.2 version has?
All in all, JM has been working for me without problems for a month now and this can be closed as far as I'm concerned.
Now getting an issue on the aarch64 RaspberryOS coming down to this cffi
version mismatch:
Exception: Version mismatch: this is the 'cffi' package version 1.14.4, located in '/tmp/pip-build-env-fch4c0zj/overlay/lib/python3.7/site-packages/cffi/api.py'. When we import the top-level '_cffi_backend' extension module, we get version 1.12.2, located in '/usr/lib/python3/dist-packages/_cffi_backend.cpython-37m-aarch64-linux-gnu.so'. The two versions should be equal; check your installation.
As it looks argon2-cffi-20.1.0
pulls a different version (v1.12.2) of cffi
than what coincurve uses (v1.14.4).
How can I pin the versions of cffi separately?
Detailed logs are linked in: https://github.com/rootzoll/raspiblitz/issues/1851#issuecomment-740598085
With a fresh install, I'm getting:
pip show cffi
Name: cffi
Version: 1.14.4
Summary: Foreign Function Interface for Python calling C code.
Home-page: http://cffi.readthedocs.org
Author: Armin Rigo, Maciej Fijalkowski
Author-email: python-cffi@googlegroups.com
License: MIT
Location: /home/waxwing/testjm2/joinmarket-clientserver/jmvenv/lib/python3.6/site-packages
Requires: pycparser
Required-by: cryptography, coincurve, argon2-cffi
(jmvenv) waxwing@here~/testjm2/joinmarket-clientserver/scripts$ pip show argon2-cffi
Name: argon2-cffi
Version: 20.1.0
Summary: The secure Argon2 password hashing algorithm.
Home-page: https://argon2-cffi.readthedocs.io/
Author: Hynek Schlawack
Author-email: hs@ox.cx
License: MIT
Location: /home/waxwing/testjm2/joinmarket-clientserver/jmvenv/lib/python3.6/site-packages
Requires: cffi, six
Required-by: joinmarketclient
(and it works of course)
So I'm not seeing "As it looks argon2-cffi-20.1.0 pulls a different version (v1.12.2) of cffi" ; so is this ARM specific or is there just some mistake somewhere? 1.14.4 seems to simply be the latest version.
For argon2-cffi doesn't this mean anything above 1.0 should be OK? https://github.com/hynek/argon2-cffi/blob/master/setup.py#L103
And in coincurve I'm seeing: https://github.com/ofek/coincurve/blob/master/setup.py#L268 , i.e. it's not pinning specifically 1.14.4 ? (as above, it's just latest I think)
Thank you for looking into it.
Managed to fix it by removing the conflicting cached file:
sudo rm /usr/lib/python3/dist-packages/_cffi_backend.cpython-37m-aarch64-linux-gnu.so
Should not be an issue on a new install.
Works well again on aarch64 (64bit ARM).
Thanks, good to hear. I'd like to understand better, though the error message/exception you got was pretty clear as to the cause of the problem, but not .. the cause of the cause of the problem if you see what I mean :) Is the problem that we are picking up a global cffi_backend module instead of installing it inside our own installation environment? (venv)
@openoms is there anything still actionable here or should we close it?
I had no installation issues with the more recent versions and some of my custom changes were addressed in other PRs so I don't feel this issue being relevant any more.
Good day, from what I've read here, I've decided to type on my terminal pip install coincurve==13.0.0 and it was successfully installed, then after that I also tried installing the pip install bit and it finally successfully installed bit too. Thank you so much!
Hi,
I'm trying to install joinmarket on:
The install script exits with the following error:
If I read this correctly, it looks like it fails while building coincurve:
I'm not sure what can I do about this and I would appriciate someone pointing me in the right direction.