Missing shared object file after upgrading to Python 3.12

Overview

I've upgraded my python venv (running homeassistant) from Python3.11.6 to Python3.12.2 today. When starting hass, I get an error message stating av cannot be imported because a shared object file is missing.

Expected behavior

import av works successfully.

Actual behavior

We try import av, but we get ImportError: libmmal_core.so.0: cannot open shared object file: No such file or directory.

Traceback:

2024-05-02 15:10:29.329 ERROR (MainThread) [homeassistant.setup] Error during setup of component stream
Traceback (most recent call last):
  File "/home/pi/test-venv/lib/python3.12/site-packages/homeassistant/setup.py", line 398, in _async_setup_component
    result = await task
             ^^^^^^^^^^
  File "/home/pi/test-venv/lib/python3.12/site-packages/homeassistant/components/stream/__init__.py", line 226, in async_setup
    await hass.async_add_executor_job(set_pyav_logging, debug_enabled)
  File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pi/test-venv/lib/python3.12/site-packages/homeassistant/components/stream/__init__.py", line 196, in set_pyav_logging
    import av  # pylint: disable=import-outside-toplevel
    ^^^^^^^^^
  File "/home/pi/test-venv/lib/python3.12/site-packages/av/__init__.py", line 20, in <module>
    from av._core import time_base, library_versions
ImportError: libmmal_core.so.0: cannot open shared object file: No such file or directory

Investigation

libmmal libraries are detected by ldconfig -p

$ ldconfig -p | grep libmmal

        libmmal_vc_client.so (libc6,hard-float) => /opt/vc/lib/libmmal_vc_client.so
        libmmal_util.so (libc6,hard-float) => /opt/vc/lib/libmmal_util.so
        libmmal_core.so (libc6,hard-float) => /opt/vc/lib/libmmal_core.so
        libmmal_components.so (libc6,hard-float) => /opt/vc/lib/libmmal_components.so
        libmmal.so (libc6,hard-float) => /opt/vc/lib/libmmal.so

symlinks from libmmal_core.so.0 to libmmal_core.so and others are present

$ ls -l /opt/vc/lib

total 3384
-rw-r--r-- 1 root root  99152 Oct 21  2023 libbcm_host.so
-rw-r--r-- 1 root root 202072 Oct 21  2023 libbrcmEGL.so
-rw-r--r-- 1 root root 105768 Oct 21  2023 libbrcmGLESv2.so
-rw-r--r-- 1 root root  99208 Oct 21  2023 libbrcmOpenVG.so
-rw-r--r-- 1 root root  78552 Oct 21  2023 libbrcmWFC.so
-rw-r--r-- 1 root root 364540 Oct 21  2023 libcontainers.so
-rw-r--r-- 1 root root  16920 Oct 21  2023 libdebug_sym.so
-rw-r--r-- 1 root root  14218 Oct 21  2023 libdebug_sym_static.a
-rw-r--r-- 1 root root  67320 Oct 21  2023 libdtovl.so
-rw-r--r-- 1 root root 202072 Oct 21  2023 libEGL.so
-rw-r--r-- 1 root root 245356 Oct 21  2023 libEGL_static.a
-rw-r--r-- 1 root root 344732 Oct 21  2023 libelftoolchain.so
lrwxrwxrwx 1 root root     12 Oct 21  2023 libGLESv1_CM.so -> libGLESv2.so
-rw-r--r-- 1 root root 105768 Oct 21  2023 libGLESv2.so
-rw-r--r-- 1 root root 100076 Oct 21  2023 libGLESv2_static.a
-rw-r--r-- 1 root root  13548 Oct 21  2023 libkhrn_client.a
-rw-r--r-- 1 root root 450544 Oct 21  2023 libkhrn_static.a
-rw-r--r-- 1 root root  68956 Oct 21  2023 libmmal_components.so
lrwxrwxrwx 1 root root     21 May  2 14:10 libmmal_components.so.0 -> libmmal_components.so
-rw-r--r-- 1 root root  82388 Oct 21  2023 libmmal_core.so
lrwxrwxrwx 1 root root     15 May  1 22:10 libmmal_core.so.0 -> libmmal_core.so
-rw-r--r-- 1 root root  15368 Oct 21  2023 libmmal.so
lrwxrwxrwx 1 root root     10 May  2 14:10 libmmal.so.0 -> libmmal.so
-rw-r--r-- 1 root root  91696 Oct 21  2023 libmmal_util.so
-rw-r--r-- 1 root root  44644 Oct 21  2023 libmmal_vc_client.so
lrwxrwxrwx 1 root root     20 May  2 14:10 libmmal_vc_client.so.0 -> libmmal_vc_client.so
-rw-r--r-- 1 root root  33208 Oct 21  2023 libopenmaxil.so
-rw-r--r-- 1 root root  99200 Oct 21  2023 libOpenVG.so
-rw-r--r-- 1 root root  30288 Oct 21  2023 libvchiq_arm.so
-rw-r--r-- 1 root root 108788 Oct 21  2023 libvchostif.a
-rw-r--r-- 1 root root  28462 Oct 21  2023 libvcilcs.a
-rw-r--r-- 1 root root 173224 Oct 21  2023 libvcos.so
-rw-r--r-- 1 root root  31672 Oct 21  2023 libvcsm.so
-rw-r--r-- 1 root root  78552 Oct 21  2023 libWFC.so
drwxr-xr-x 2 root root   4096 Oct 21  2023 pkgconfig
drwxr-xr-x 2 root root   4096 Oct 21  2023 plugins

Library directory is included for ldconfig:

$ cat /etc/ld.so.conf.d/00-vmcs.conf 

/opt/vc/lib

Reproduction

(Try to use homeassistant>=2024.4 with the stream integration, which needs ha-av, a fork of av. When force using plain av==12.0.0 the issue persists. Or short version:) pip install av or pip install av --no-binary av, then try to import av.

Versions

OS: uname -a: Linux raspi4 6.1.58-v8+ #1692 SMP PREEMPT Thu Oct 19 17:06:12 BST 2023 aarch64 GNU/Linux
PyAV runtime:
```
$ python -m av --version
```

Traceback (most recent call last): File "", line 189, in _run_module_as_main File "", line 148, in _get_module_details File "", line 112, in _get_module_details File "/home/pi/test-venv/lib/python3.12/site-packages/av/init.py", line 20, in from av._core import time_base, library_versions ImportError: libmmal_core.so.0: cannot open shared object file: No such file or directory

- PyAV build:

$ pip3 install --force-reinstall --no-cache-dir av -vvv av --no-binary av


[output.log](https://github.com/PyAV-Org/PyAV/files/15190580/output.log)

## Research

I have done the following:

- [x] Checked the [PyAV documentation](https://pyav.basswood-io.com)
- [x] Searched on [Google](https://www.google.com/search?q=pyav+how+do+I+foo)
- [x] Searched on [Stack Overflow](https://stackoverflow.com/search?q=pyav)
- [x] Looked through [old GitHub issues](https://github.com/PyAV-Org/PyAV/issues?&q=is%3Aissue)

## Additional context

There was no issue running `homeassistant==2024.3.1` with Python3.11.6 and `ha-av==10.1.1` which is built using `av==10.0.0` ( https://github.com/uvjustin/PyAV/releases/tag/v10.1.1 ).
The current version I'm trying to run is also `ha-av==10.1.1` but now with Python3.12.2 and `homeassistant==2024.4.4`. I also tried plain `av==12.0.0` but to no avail (output.log is from building `av==12.0.0`)
Pythhon3.12.2 I had to compile myself as there is no package. I downloaded the .tar.gz, `./configure --enable-optimizations`, `make -j4` and `sudo make altinstall`. Then I created a new venv.

PyAV-Org / PyAV