Problem with installing from wheels on Raspberry PI

[vsajip]

I have a problem when installing cffi from wheels on Python 3.9 and later. Here is my environment:

pi@rpi2:~ $ more /etc/os-release
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"

Here is a console session that demonstrates:

pi@rpi2:~ $ mkdir /tmp/cffitest
pi@rpi2:~ $ cd /tmp/cffitest
pi@rpi2:/tmp/cffitest $ python3.9 -m venv venv
pi@rpi2:/tmp/cffitest $ venv/bin/pip install cffi
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting cffi
  Using cached https://www.piwheels.org/simple/cffi/cffi-1.16.0-cp39-cp39-linux_armv7l.whl (367 kB)
Collecting pycparser
  Using cached https://www.piwheels.org/simple/pycparser/pycparser-2.21-py2.py3-none-any.whl (119 kB)
Installing collected packages: pycparser, cffi
Successfully installed cffi-1.16.0 pycparser-2.21

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: python3.9 -m pip install --upgrade pip

At this point cffi appears to have been installed, but all is not well: trying to import _cffi_backend fails:

pi@rpi2:/tmp/cffitest $ venv/bin/python
Python 3.9.18 (main, Nov  9 2023, 13:01:52)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _cffi_backend as backend
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: libffi.so.7: cannot open shared object file: No such file or directory
>>>

If I uninstall cffi and reinstall specifying that a source build should be done, there is no import problem with the reinstalled cffi.

pi@rpi2:/tmp/cffitest $ venv/bin/pip uninstall cffi
Found existing installation: cffi 1.16.0
Uninstalling cffi-1.16.0:
  Would remove:
    /tmp/cffitest/venv/lib/python3.9/site-packages/_cffi_backend.cpython-39-arm-linux-gnueabihf.so
    /tmp/cffitest/venv/lib/python3.9/site-packages/cffi-1.16.0.dist-info/*
    /tmp/cffitest/venv/lib/python3.9/site-packages/cffi/*
Proceed (Y/n)? y
  Successfully uninstalled cffi-1.16.0
pi@rpi2:/tmp/cffitest $ venv/bin/pip install --no-binary cffi cffi
DEPRECATION: --no-binary currently disables reading from the cache of locally built wheels. In the future --no-binary will not influence the wheel cache. pip 23.1 will enforce this behaviour change. A possible replacement is to use the --no-cache-dir option. You can use the flag --use-feature=no-binary-enable-wheel-cache to test the upcoming behaviour. Discussion can be found at https://github.com/pypa/pip/issues/11453
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting cffi
  Using cached cffi-1.16.0.tar.gz (512 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pycparser in ./venv/lib/python3.9/site-packages (from cffi) (2.21)
Building wheels for collected packages: cffi
  Building wheel for cffi (pyproject.toml) ... done
  Created wheel for cffi: filename=cffi-1.16.0-cp39-cp39-linux_armv7l.whl size=404098 sha256=32fdbada732b7c30e1efe746d49a9aa319f908271f74892a074694cd4fe7755d
  Stored in directory: /home/pi/.cache/pip/wheels/49/dd/bc/ce13506ca37e8052ad8f14cdfc175db1b3943e5d7bce7718e3
Successfully built cffi
Installing collected packages: cffi
Successfully installed cffi-1.16.0

[notice] A new release of pip is available: 23.0.1 -> 23.3.1
[notice] To update, run: python3.9 -m pip install --upgrade pip
pi@rpi2:/tmp/cffitest $ venv/bin/python
Python 3.9.18 (main, Nov  9 2023, 13:01:52)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import _cffi_backend as backend
>>>

I've observed the same problem (and with the same workaround) with Python 3.10, 3.11 and 3.12 - but the problem doesn't seem to occur with Python 3.8 and earlier.

[mattip]

According to their page, you need bullseye and or bookworm for python>3.8 but you are running buster. Either update your OS version or stick with the older python versions.

[vsajip]

Ah OK, thanks - I wasn't sure who was producing these wheels. I need to support later Python versions for building some extensions which aren't OS-level dependent, so I'm fine with building cffi from source. Of course you can't tell OS-level dependencies from a wheel other than the machine architecture and base OS - another small glitch in the Python packaging ecosystem! I'll close this issue now.

[mattip]

you can't tell OS-level dependencies from a wheel

You can on wheels produced using the manylinux standard. But a manylinux standard for armv7 never took off. If you move to armv8 (64 bits), you can use the aarch64 wheels from PyPI.

[vsajip]

Maybe when I get a Raspberry Pi 5, I'll do that :smile: