Poetry install is failing

[winksaville]

Describe the bug I'm running on an Arch Linux system and running the poetry install command from here fails:

wink@3900x 24-03-17T02:54:55.125Z:~/prgs/clones/trezor-firmware (main)
$ poetry install
Creating virtualenv trezor-firmware in /home/wink/prgs/clones/trezor-firmware/.venv
Installing dependencies from lock file

Package operations: 88 installs, 1 update, 0 removals

  - Installing attrs (21.4.0)
  - Installing certifi (2023.7.22)

...

  - Installing pyasn1 (0.4.8)
  - Installing pyblake2 (1.1.2): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel

  running bdist_wheel
  running build
  running build_ext
  building 'pyblake2' extension
  creating build
  creating build/temp.linux-x86_64-cpython-311
  gcc -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -DBLAKE2_COMPRESS_AUTO=1 -I/tmp/tmpm4qu9khe/.venv/include -I/usr/include/python3.11 -c blake2b_impl.c -o build/temp.linux-x86_64-cpython-311/blake2b_impl.o
  gcc -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -DBLAKE2_COMPRESS_AUTO=1 -I/tmp/tmpm4qu9khe/.venv/include -I/usr/include/python3.11 -c blake2s_impl.c -o build/temp.linux-x86_64-cpython-311/blake2s_impl.o
  gcc -DNDEBUG -g -fwrapv -O3 -Wall -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -ffat-lto-objects -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -march=x86-64 -mtune=generic -O3 -pipe -fno-plt -fexceptions -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security -fstack-clash-protection -fcf-protection -g -ffile-prefix-map=/build/python/src=/usr/src/debug/python -flto=auto -fPIC -DBLAKE2_COMPRESS_AUTO=1 -I/tmp/tmpm4qu9khe/.venv/include -I/usr/include/python3.11 -c pyblake2module.c -o build/temp.linux-x86_64-cpython-311/pyblake2module.o
  pyblake2module.c: In function ‘PyInit_pyblake2’:
  pyblake2module.c:699:27: error: lvalue required as left operand of assignment
    699 |     Py_TYPE(&blake2bType) = &PyType_Type;
        |                           ^
  pyblake2module.c:703:27: error: lvalue required as left operand of assignment
    703 |     Py_TYPE(&blake2sType) = &PyType_Type;
        |                           ^
  error: command '/usr/bin/gcc' failed with exit code 1

  at ~/.local/share/pipx/venvs/poetry/lib/python3.11/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with pyblake2 (1.1.2) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "pyblake2 (==1.1.2)"'.

  - Installing pyserial (3.5)
  - Installing pytest-ordering (0.6)
  - Installing pytest-random-order (1.0.4)
  - Installing pytest-timeout (2.1.0)
  - Installing pytest-xdist (2.5.0)
  - Installing python-bitcoinlib (0.11.0)
  - Installing scan-build (2.0.20)
  - Installing scons (4.3.0)
  - Installing shamir-mnemonic (0.2.2)
  - Installing towncrier (23.6.0)
  - Installing vulture (2.6)
  - Installing yamllint (1.26.3)
  - Installing tox (3.24.5)
  - Installing toiftool (0.1.0 /home/wink/prgs/clones/trezor-firmware/python/tools/toiftool)
  - Installing trezor-pylint-plugin (0.1.0 /home/wink/prgs/clones/trezor-firmware/tools/trezor-pylint-plugin)
wink@3900x 24-03-17T02:55:08.294Z:~/prgs/clones/trezor-firmware (main)
$ 

Firmware version and revision This is main as of March 17 2024, here is tip of tree:

wink@3900x 24-03-17T03:22:50.844Z:~/prgs/clones/trezor-firmware (main)
$ git log -1
commit 6e1cb9ff8c5907382ce0daeece1d4c6ba38d067d (HEAD -> main, origin/main, origin/HEAD)
Author: Martin Milata <martin@martinmilata.cz>
Date:   Fri Mar 15 14:55:11 2024 +0100

    tests: update fixtures.json
wink@3900x 24-03-17T03:26:57.788Z:~/prgs/clones/trezor-firmware (main)

Desktop/smartphone setup (please complete the following information):

• Wallet software: trezor-firmware
• OS: Arch Linux

  wink@3900x 24-03-17T03:26:57.788Z:~/prgs/clones/trezor-firmware (main)
  $ inxi
  CPU: 12-core AMD Ryzen 9 3900X (-MT MCP-) speed/min/max: 2370/2200/4672 MHz
  Kernel: 6.8.1-arch1-1 x86_64 Up: 2h 49m Mem: 6.41/31.27 GiB (20.5%)
  Storage: 1.82 TiB (34.0% used) Procs: 495 Shell: Bash inxi: 3.3.33
  wink@3900x 24-03-17T03:29:30.545Z:~/prgs/clones/trezor-firmware (main)

To Reproduce Steps to reproduce the behavior:

1. git clone --recurse-submodules https://github.com/trezor/trezor-firmware.git
2. cd trezor-firmware
3. poetry install
4. See error

Expected behavior I expect to successfully run poetry install

Screenshots Output from terminal CLI executions are above

Additional context

wink@3900x 24-03-17T03:29:30.545Z:~/prgs/clones/trezor-firmware (main)
$ git --version
git version 2.44.0
wink@3900x 24-03-17T03:32:34.596Z:~/prgs/clones/trezor-firmware (main)
$ python3 --version
Python 3.11.8
wink@3900x 24-03-17T03:32:44.997Z:~/prgs/clones/trezor-firmware (main)
$ pipx --version
1.4.3
wink@3900x 24-03-17T03:32:49.319Z:~/prgs/clones/trezor-firmware (main)
$ poetry --version
Poetry (version 1.8.2)
wink@3900x 24-03-17T03:32:59.605Z:~/prgs/clones/trezor-firmware (main)
$ 

[winksaville]

Sometime soon it would seem pyblake2 should not be used, it has been archived since Sept. 14, 2020 as Python 3.6 and later have BLAKE2 in hashlib, see here.

[winksaville]

I was able to fix this by forking pyblake2. Creating a branch, https://github.com/winksaville/pyblake2/tree/change-Py_TYPE-to-Py_SET_TYPE-for-gcc-3.11 and adding this commit, https://github.com/winksaville/pyblake2/commit/b32d7fc2d16330fcefda9c8eba217c80949dd7d2. With this trivial change poetry install --no-root works.

But, as I mention above, since pyblake2 is archived it would probably be a good idea to use a different solution.

[matejcik]

i mean, as a workaround i'm pretty sure you can just remove pyblake from pyproject.toml

[winksaville]

Removing pyblake2 since it's read-only is the right fix, #3637 does that.