search

matrix-org / pantalaimon

E2EE aware proxy daemon for matrix clients.
Apache License 2.0
292 stars 42 forks source link

Build pantalaimon for docker on raspberry pi 4 openssl gcc errror. #76

Open Sleuth56 opened 3 years ago

Sleuth56 commented 3 years ago

Describe the bug openssl encryption wheel failing to build pantalaimon while using Docker on armhf based distro.

To Reproduce Steps to reproduce the behavior:

  1. Get your raspi out with rasbian
  2. Clone pantalaimon with git
  3. Run the docker build command from the readme.md
  4. Wait until it gets to building the wheels.
  5. See error

Expected behavior To build without problems.

Desktop (please complete the following information):

Additional context Using the same exact commands built and ran just fine on my manjaro X86 desktop.

Error message Building wheel for cryptography (PEP 517): started Building wheel for cryptography (PEP 517): finished with status 'error' ERROR: Command errored out with exit status 1: command: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpiuapdt_0 cwd: /tmp/pip-wheel-ttnk1y45/cryptography Complete output (137 lines): running bdist_wheel running build running build_py creating build creating build/lib.linux-armv7l-3.8 creating build/lib.linux-armv7l-3.8/cryptography copying src/cryptography/exceptions.py -> build/lib.linux-armv7l-3.8/cryptography copying src/cryptography/fernet.py -> build/lib.linux-armv7l-3.8/cryptography copying src/cryptography/utils.py -> build/lib.linux-armv7l-3.8/cryptography copying src/cryptography/__init__.py -> build/lib.linux-armv7l-3.8/cryptography copying src/cryptography/__about__.py -> build/lib.linux-armv7l-3.8/cryptography creating build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/oid.py -> build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/ocsp.py -> build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/certificate_transparency.py -> build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/base.py -> build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/name.py -> build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/general_name.py -> build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/extensions.py -> build/lib.linux-armv7l-3.8/cryptography/x509 copying src/cryptography/x509/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/x509 creating build/lib.linux-armv7l-3.8/cryptography/hazmat copying src/cryptography/hazmat/_der.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat copying src/cryptography/hazmat/_oid.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat copying src/cryptography/hazmat/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat creating build/lib.linux-armv7l-3.8/cryptography/hazmat/bindings copying src/cryptography/hazmat/bindings/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/bindings creating build/lib.linux-armv7l-3.8/cryptography/hazmat/backends copying src/cryptography/hazmat/backends/interfaces.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends copying src/cryptography/hazmat/backends/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends creating build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/poly1305.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/hashes.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/keywrap.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/hmac.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/padding.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/cmac.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives copying src/cryptography/hazmat/primitives/constant_time.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives creating build/lib.linux-armv7l-3.8/cryptography/hazmat/bindings/openssl copying src/cryptography/hazmat/bindings/openssl/binding.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/bindings/openssl copying src/cryptography/hazmat/bindings/openssl/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/bindings/openssl copying src/cryptography/hazmat/bindings/openssl/_conditional.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/bindings/openssl creating build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/x448.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/dh.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/decode_asn1.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ciphers.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/aead.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/dsa.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ocsp.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/x509.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ed448.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/poly1305.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/hashes.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ec.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/hmac.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/encode_asn1.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/backend.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/rsa.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/cmac.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/x25519.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/utils.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl copying src/cryptography/hazmat/backends/openssl/ed25519.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/backends/openssl creating build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/x448.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/dh.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/dsa.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/ed448.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/ec.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/padding.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/rsa.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/x25519.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/utils.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric copying src/cryptography/hazmat/primitives/asymmetric/ed25519.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/asymmetric creating build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/totp.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/hotp.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/utils.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/twofactor copying src/cryptography/hazmat/primitives/twofactor/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/twofactor creating build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/pkcs7.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/base.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/pkcs12.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/serialization copying src/cryptography/hazmat/primitives/serialization/ssh.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/serialization creating build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/modes.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/aead.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/base.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/algorithms.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/ciphers copying src/cryptography/hazmat/primitives/ciphers/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/ciphers creating build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/scrypt.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/concatkdf.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/pbkdf2.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/hkdf.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/kbkdf.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/__init__.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf copying src/cryptography/hazmat/primitives/kdf/x963kdf.py -> build/lib.linux-armv7l-3.8/cryptography/hazmat/primitives/kdf running egg_info writing src/cryptography.egg-info/PKG-INFO writing dependency_links to src/cryptography.egg-info/dependency_links.txt writing requirements to src/cryptography.egg-info/requires.txt writing top-level names to src/cryptography.egg-info/top_level.txt reading manifest file 'src/cryptography.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' no previously-included directories found matching 'docs/_build' warning: no previously-included files found matching 'vectors' warning: no previously-included files matching '*' found under directory 'vectors' warning: no previously-included files found matching '.travis.yml' warning: no previously-included files found matching '.travis' warning: no previously-included files matching '*' found under directory '.travis' warning: no previously-included files matching '*' found under directory '.github' warning: no previously-included files found matching 'release.py' warning: no previously-included files found matching '.coveragerc' warning: no previously-included files found matching 'codecov.yml' warning: no previously-included files found matching 'dev-requirements.txt' warning: no previously-included files found matching 'rtd-requirements.txt' warning: no previously-included files found matching 'tox.ini' warning: no previously-included files matching '*' found under directory '.zuul.d' warning: no previously-included files matching '*' found under directory '.zuul.playbooks' writing manifest file 'src/cryptography.egg-info/SOURCES.txt' running build_ext generating cffi module 'build/temp.linux-armv7l-3.8/_padding.c' creating build/temp.linux-armv7l-3.8 generating cffi module 'build/temp.linux-armv7l-3.8/_openssl.c' building '_openssl' extension creating build/temp.linux-armv7l-3.8/build creating build/temp.linux-armv7l-3.8/build/temp.linux-armv7l-3.8 gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/usr/local/include/python3.8 -c build/temp.linux-armv7l-3.8/_openssl.c -o build/temp.linux-armv7l-3.8/build/temp.linux-armv7l-3.8/_openssl.o -Wconversion -Wno-error=sign-conversion build/temp.linux-armv7l-3.8/_openssl.c:575:10: fatal error: openssl/opensslv.h: No such file or directory #include ^~~~~~~~~~~~~~~~~~~~ compilation terminated. error: command 'gcc' failed with exit status 1 ---------------------------------------- ERROR: Failed building wheel for cryptography Building wheel for pyrsistent (setup.py): started Building wheel for pyrsistent (setup.py): finished with status 'done' Created wheel for pyrsistent: filename=pyrsistent-0.17.3-cp38-cp38-linux_armv7l.whl size=68191 sha256=baafb8bd804d33109d7460623be783893b00859f128f0b07bb76434f99a397c5 Stored in directory: /root/.cache/pip/wheels/3d/22/08/7042eb6309c650c7b53615d5df5cc61f1ea9680e7edd3a08d2 Successfully built pantalaimon aiohttp logbook peewee multidict yarl pycryptodome future python-olm pyrsistent Failed to build cryptography ERROR: Failed to build one or more wheels The command '/bin/sh -c pip wheel . --wheel-dir /wheels --find-links /wheels' returned a non-zero code: 1
poljar commented 3 years ago

That's an error in the OpenSSL package and I have no idea why that is failing, the OpenSSL package might have prebuild binary wheels for X86 but not ARM, perhaps some header files for OpenSSL are needed to make the build succeed.

HidingCherry commented 3 years ago

This is weird, I have built it today (arm64) and I didn't run into that issue. But I use archlinuxarm and I have set up a docker-compose file, so I don't have to redo the commands. I also don't have a raspberry pi, I just want to show that it actually builds on arm64 (not armhf, sorry).

Here is the part where wheels is being build, if that helps in any way:

Building wheels for collected packages: pantalaimon, logbook, peewee, future, python-olm, pyrsistent
  Building wheel for pantalaimon (setup.py): started
  Building wheel for pantalaimon (setup.py): finished with status 'done'
  Created wheel for pantalaimon: filename=pantalaimon-0.8.0-py3-none-any.whl size=45992 sha256=3152933b7787713ecd88e88d898b71f0caf764e3d0dec7647419e1fc3f8e30d7
  Stored in directory: /tmp/pip-ephem-wheel-cache-r9bgwr_8/wheels/40/5d/89/f38b7a58cbc6f6bdff423c409ca06a097d4abae527d8c17f06
  Building wheel for logbook (setup.py): started
  Building wheel for logbook (setup.py): finished with status 'done'
  Created wheel for logbook: filename=Logbook-1.5.3-cp38-cp38-linux_aarch64.whl size=66381 sha256=56c466f9b616bddf4a416de8c1d9246a3de8ea15fe23112b06d28d841733e99a
  Stored in directory: /root/.cache/pip/wheels/25/9a/c0/03bf368ac5f673e9c0bde58837146b004424b25555ca3aedfb
  Building wheel for peewee (setup.py): started
  Building wheel for peewee (setup.py): finished with status 'done'
  Created wheel for peewee: filename=peewee-3.14.0-py3-none-any.whl size=128142 sha256=7dcbf12e8c726fc21ffbfdba119c0b4ad77773330cd16712eaa7c5bb3819eb1b
  Stored in directory: /root/.cache/pip/wheels/f9/0c/4f/37984c77779105edbafffd114ac40820bfa4f1f1bf9ff6ed57
  Building wheel for future (setup.py): started
  Building wheel for future (setup.py): finished with status 'done'
  Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491059 sha256=9ac9bd61bb2c687baa0d61c92f58fd16bcb2086c6a346d2d2680d4dde397f95b
  Stored in directory: /root/.cache/pip/wheels/8e/70/28/3d6ccd6e315f65f245da085482a2e1c7d14b90b30f239e2cf4
  Building wheel for python-olm (setup.py): started
  Building wheel for python-olm (setup.py): finished with status 'done'
  Created wheel for python-olm: filename=python_olm-3.1.3-cp38-cp38-linux_aarch64.whl size=47601 sha256=8cfd5021360b8a344fafa257ac74eac74c37b3b87fc94aebaa9946c2037bec27
  Stored in directory: /root/.cache/pip/wheels/1c/a0/53/ea4c79f60693fcef0951216bb94ee25545632e686403fbb438
  Building wheel for pyrsistent (setup.py): started
  Building wheel for pyrsistent (setup.py): finished with status 'done'
  Created wheel for pyrsistent: filename=pyrsistent-0.17.3-cp38-cp38-linux_aarch64.whl size=71601 sha256=d73901b4d47a711ca2fc5ae91181291f2182a11c9568d656761695a4670a3f91
  Stored in directory: /root/.cache/pip/wheels/3d/22/08/7042eb6309c650c7b53615d5df5cc61f1ea9680e7edd3a08d2
Successfully built pantalaimon logbook peewee future python-olm pyrsistent
Removing intermediate container f8975a76d25f
 ---> 299bffdc9616

Step 9/16 : FROM python:3.8-slim-buster AS run
 ---> de695ed9e8c4
Step 10/16 : COPY --from=builder /usr/lib/libolm* /usr/lib/
 ---> 2d3f399c2bf0
Step 11/16 : COPY --from=builder /wheels /wheels
 ---> a871a7ea0d23
Step 12/16 : WORKDIR /app
 ---> Running in 824b288405f8
Removing intermediate container 824b288405f8
 ---> 17ba3e92842c
Step 13/16 : RUN pip --no-cache-dir install --find-links /wheels --no-index pantalaimon
 ---> Running in 27688fc3e52c
Looking in links: /wheels
Processing /wheels/pantalaimon-0.8.0-py3-none-any.whl
Processing /wheels/matrix_nio-0.15.2-py3-none-any.whl
Processing /wheels/peewee-3.14.0-py3-none-any.whl
Processing /wheels/aiohttp-3.7.3-cp38-cp38-manylinux2014_aarch64.whl
Processing /wheels/attrs-20.3.0-py2.py3-none-any.whl
Processing /wheels/cachetools-4.1.1-py3-none-any.whl
Processing /wheels/click-7.1.2-py2.py3-none-any.whl
Processing /wheels/Logbook-1.5.3-cp38-cp38-linux_aarch64.whl
Processing /wheels/keyring-21.5.0-py3-none-any.whl
Processing /wheels/janus-0.6.1-py3-none-any.whl
Processing /wheels/appdirs-1.4.4-py2.py3-none-any.whl
Processing /wheels/aiofiles-0.4.0-py3-none-any.whl
Processing /wheels/jsonschema-3.2.0-py2.py3-none-any.whl
Processing /wheels/future-0.18.2-py3-none-any.whl
Processing /wheels/h2-3.2.0-py2.py3-none-any.whl
Processing /wheels/pycryptodome-3.9.9-cp38-cp38-manylinux2014_aarch64.whl
Processing /wheels/h11-0.9.0-py2.py3-none-any.whl
Processing /wheels/unpaddedbase64-1.1.0-py2.py3-none-any.whl
Processing /wheels/python_olm-3.1.3-cp38-cp38-linux_aarch64.whl
Processing /wheels/atomicwrites-1.4.0-py2.py3-none-any.whl
Processing /wheels/async_timeout-3.0.1-py3-none-any.whl
Processing /wheels/multidict-5.0.2-cp38-cp38-manylinux2014_aarch64.whl
Processing /wheels/yarl-1.6.3-cp38-cp38-manylinux2014_aarch64.whl
Processing /wheels/chardet-3.0.4-py2.py3-none-any.whl
Processing /wheels/typing_extensions-3.7.4.3-py3-none-any.whl
Processing /wheels/SecretStorage-3.3.0-py3-none-any.whl
Processing /wheels/jeepney-0.6.0-py3-none-any.whl
Processing /wheels/six-1.15.0-py2.py3-none-any.whl
Processing /wheels/pyrsistent-0.17.3-cp38-cp38-linux_aarch64.whl
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/site-packages (from jsonschema<4.0.0,>=3.2.0->matrix-nio[e2e]<0.16,>=0.14->pantalaimon) (50.3.2)
Processing /wheels/hyperframe-5.2.0-py2.py3-none-any.whl
Processing /wheels/hpack-3.0.0-py2.py3-none-any.whl
Processing /wheels/cffi-1.14.4-cp38-cp38-linux_aarch64.whl
Processing /wheels/idna-2.10-py2.py3-none-any.whl
Processing /wheels/cryptography-3.2.1-cp35-abi3-manylinux2014_aarch64.whl
Processing /wheels/pycparser-2.20-py2.py3-none-any.whl
Installing collected packages: async-timeout, multidict, idna, yarl, chardet, attrs, typing-extensions, aiohttp, aiofiles, six, pyrsistent, jsonschema, future, hyperframe, hpack, h2, pycryptodome, logbook, h11, unpaddedbase64, pycparser, cffi, python-olm, peewee, atomicwrites, cachetools, matrix-nio, click, cryptography, jeepney, SecretStorage, keyring, janus, appdirs, pantalaimon
Successfully installed SecretStorage-3.3.0 aiofiles-0.4.0 aiohttp-3.7.3 appdirs-1.4.4 async-timeout-3.0.1 atomicwrites-1.4.0 attrs-20.3.0 cachetools-4.1.1 cffi-1.14.4 chardet-3.0.4 click-7.1.2 cryptography-3.2.1 future-0.18.2 h11-0.9.0 h2-3.2.0 hpack-3.0.0 hyperframe-5.2.0 idna-2.10 janus-0.6.1 jeepney-0.6.0 jsonschema-3.2.0 keyring-21.5.0 logbook-1.5.3 matrix-nio-0.15.2 multidict-5.0.2 pantalaimon-0.8.0 peewee-3.14.0 pycparser-2.20 pycryptodome-3.9.9 pyrsistent-0.17.3 python-olm-3.1.3 six-1.15.0 typing-extensions-3.7.4.3 unpaddedbase64-1.1.0 yarl-1.6.3
Removing intermediate container 27688fc3e52c
 ---> e3d9ab9a6184
Step 14/16 : VOLUME /data
 ---> Running in 2c0c6f90f06c
Removing intermediate container 2c0c6f90f06c
 ---> a9008863cc53
Step 15/16 : ENTRYPOINT ["pantalaimon"]
 ---> Running in bcb416546426
Removing intermediate container bcb416546426
 ---> 2a7d1529396f
Step 16/16 : CMD ["-c", "/data/pantalaimon.conf", "--data-path", "/data"]
 ---> Running in 3c6a790ea9cf
Removing intermediate container 3c6a790ea9cf
 ---> 000e19484da8

Successfully built 000e19484da8
Successfully tagged matrixdotorg/pantalaimon:v0.8.0-1
WARNING: Image for service pantalaimon was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating docker_pantalaimon_run ... done
Sleuth56 commented 3 years ago

I also got Pantalaimon to build on manjaro arch arm64. On my Pinebook Pro. That makes this not a X86 problem. I think one of the dependencies that gets pulled in has an arm64 build but not an armhf one.

sunrisepi commented 3 years ago

Success! I was able to determine how to build the Docker image for Pantalaimon on Raspberry Pi OS (buster release), 32-bit (armhf). My Dockerfile update is below. @poljar, you were right, some OpenSSL files were missing. Here are the steps I followed:

- It seemed like the Dockerfile already had a command where pip was being upgraded...
`RUN pip --no-cache-dir install` **--upgrade pip** `setuptools wheel`
- ...so I decided to try installing Rust, as it seemed not to be present in the build.  A reference on [Building cryptography on Linux ](https://cryptography.io/en/latest/installation.html#building-cryptography-on-linux) was helpful.  So was [this one ](https://cryptography.io/en/latest/faq.html#installing-cryptography-fails-with-error-can-not-find-rust-compiler) on the specific error we were seeing (can't find Rust compiler), which also suggested installing Rust.
- I found the command for installing Rust on Linux [here](https://forge.rust-lang.org/infra/other-installation-methods.html#other-ways-to-install-rustup): **curl https://sh.rustup.rs -sSf | sh**
- I discovered I had to install curl as well, so added that to the Dockerfile. NOTE:  I just noticed that **wget** is already installed in the Dockerfile.. So we could use wget instead of curl.  I haven't tested that but something like **wget https://sh.rustup.rs -O /app/rustup-init.sh** should work fine.  If we want to use wget then just remove the curl installation and use wget instead of curl.
- Modified the Dockerfile to install Rust, but building of the cryptography wheel was still failing.  I then realized there was [this note ](https://www.rust-lang.org/tools/install) about checking the version of rust installed, to confirm it was in the path.  This was working initially during installation but by the time Docker build got to the step where it was trying to build the cryptography wheel, the path vanished.  
- I suspected I had to set the environment variable to ensure Rust is in the path and findable, at the same step where pip is called to build all the wheels. I had to do some research to find out how to do that.
- While reviewing the build log I noticed it mentioned this command at the end of Rust installation, to set the environment: **source $HOME/.cargo/env**, which is actually **. $HOME/.cargo/env** to "source" or refresh the environment variable for Rust.
- I then had to tweak the way I ran the Rust installer.  First had to download it via curl, saving the installation script to the /app dir, and using chmod to make it executable before running the shell script with defaults (**-y** parameter) to install Rust.  
- At first I didn't think it worked because after installing Rust, there's nearly a 20 second delay until Docker build goes to the next step.  Not sure if that is part of the Rust installation or if the Docker build process is just a bit slow on my Raspberry Pi 4b.  But it finally goes to the next step and ends up completing successfully.
- After all that I finally got a Docker image built!  It's 131MB; wondering if it's a bit too big.

I noticed after building the image there was a leftover image of 1.55GB in size with a **REPOSITORY** of **\<none\>**.  A temporary image.  Here's a summary of how I cleaned that up.  I know the other grep for the specific image GUID (9f59*) is not needed, but I was just being very cautious.

pi@pi:~/gitcode/pantalaimon $ docker images |grep none

9f5919181d71 2 minutes ago 1.55GB pi@pi:~/gitcode/pantalaimon $ docker images |grep none | awk '{print $3}' 9f5919181d71 pi@pi:~/gitcode/pantalaimon $ docker images |grep none | awk '{print $3}' | grep 9f5919181d71 | xargs docker rmi Deleted: sha256:9f5919181d71cdfb112e78613ae1b4787b33d8e60265e1fe762666874f363029 Deleted: sha256:2c8e88561289a96087ce361d46fcc23dfb948f2a444d94260f04b7b0a76d4d76 Deleted: sha256:13c80672ba8d8a225509674af7add7a97a559bd1455df7c5d3296e01761d13c8 Deleted: sha256:0d7f1485fc3af847f653f4252f37a655499fe2431ac33a61c743bb5f4e084fcd Deleted: sha256:fb6663b4d6d464e7514f788e6ac12496fee56029af97e98ac99cdb3ab4c1df8d Deleted: sha256:abb95f652f6876b3c813a14409c3663508a9069c48c09cd47de65ce953ef560a Deleted: sha256:2886a534c71b44e3c0685c99e19e20cb7d86d993de98a42fd74afb7bcb5d5775 Deleted: sha256:dc1b26208d5a7cebd3c8d980569354ecb92aab859b04e56a98a0a5a216038ad9 Deleted: sha256:2c83eea0d0f35baea56a06e76a81efaf4b8426e24a9648bb19538a7982db8ce2 Deleted: sha256:df8e8033827aa609e1d72a65fdcced63dcb22325711fa7d8e9749bb0a96848ef Deleted: sha256:097762a3bdd81a15f08737dcd41391c561d42396a368312067d1372decc5ec58 Deleted: sha256:7009ed3afa1e02cd15528fec7e18cec3b1bab85a11c9bc57a67d72d9a4e10e29 Deleted: sha256:1341375a573ee31cfb0b809d6da86148807645165728ddff6898af89a37b5c14 Deleted: sha256:5fcbd9dc4f0fe3900e57e06c694c52f9d092cf1273933c2328a05b574a34a714 Deleted: sha256:0d558c62c4d4cb9a85b66f9d29196a03441fa3b47107cbc705f0917e127af400 pi@pi:~/gitcode/pantalaimon $ docker images |grep none pi@pi:~/gitcode/pantalaimon $ docker images | grep pant pantalaimon latest b3246438542e 2 minutes ago 131MB pi@pi:~/gitcode/pantalaimon $ ``` I have the following questions; let me know if anyone has more info on these: - How do we make the image smaller? I'm not sure if 131MB is the smallest it could be. - If someone else with a Raspbery Pi/ARM has a chance to test the image created using the below Dockerfile and confirm it works, that would be great. I am hoping to do that in the next several days but it may take some time for me to get to it. I suspect it works but am just barely getting familiar with Pantalaimon and am not yet sure how to use it. - Could we also get the cryptography wheel installed with a pip upgrade? Maybe pip upgrade needs to be defined differently in the Dockerfile? I would prefer to do it that way, as then a precompiled cryptography wheel could be used instead (and that may also help make the Docker image smaller). - Has anyone used pantalaimon with PushBits, the self-hosted relay server for Matrix push notifications? I just got PushBits installed and was hoping to use pantalaimon with PushBits, so I could send end-to-end encrypted messages to my Matrix server. PushBits works great now but it doesn't support encryption. [The PushBits Github](https://github.com/pushbits/server) has more details on PushBits. I actually learned about Pantalaimon from the Matrix FAQ on [end-to-end encryption proxies.](https://matrix.org/faq/#encryption) - If anyone has further suggestions on the Dockerfile, please let me know (my findings are below). Here is the Docker file I used to build this on Raspberry Pi 32bit (armhf). Hope this helps others! I learned a lot about building a Docker image. :) I have designated **in bold text** the changes I made. Thanks. > FROM python:3.8-slim-buster AS builder > > RUN mkdir -p /app > RUN apt-get update && apt-get install -y git gcc clang cmake g++ pkg-config python3-dev wget **libssl-dev curl** > > **RUN curl https://sh.rustup.rs -o /app/rustup-init.sh && chmod +x /app/rustup-init.sh && /app/rustup-init.sh -y** > > WORKDIR /app > RUN wget https://gitlab.matrix.org/matrix-org/olm/-/archive/master/olm-master.tar.bz2 \ > && tar -xvf olm-master.tar.bz2 \ > && cd olm-master && make && make PREFIX="/usr" install > > RUN pip --no-cache-dir install --upgrade pip setuptools wheel > > COPY . /app > > RUN **. $HOME/.cargo/env &&** pip wheel . --wheel-dir /wheels --find-links /wheels > > FROM python:3.8-slim-buster AS run > > COPY --from=builder /usr/lib/libolm* /usr/lib/ > COPY --from=builder /wheels /wheels > WORKDIR /app > > RUN pip --no-cache-dir install --find-links /wheels --no-index pantalaimon > > VOLUME /data > ENTRYPOINT ["pantalaimon"] > CMD ["-c", "/data/pantalaimon.conf", "--data-path", "/data"] > >
sunrisepi commented 3 years ago

FYI, I had a chance to test the Pantalaimon Docker image I built on 04/04. When I use a web browser to visit my local IP (the server on which docker and Pantalaimon are installed) with the running Pantalaimon port, it works fine and shows Matrix.org content (which shows it is routing through Pantalaimon/acting as a proxy). Thank you.