SpotlightKid / python-rtmidi

Python bindings for the cross-platform MIDI I/O library RtMidi
https://spotlightkid.github.io/python-rtmidi/
Other
349 stars 65 forks source link

MidiInAlsa::initialize: error creating ALSA sequencer client object #138

Closed Bubobubobubobubo closed 8 months ago

Bubobubobubobubo commented 1 year ago

Hi,

I opened a mirror issue in the mido repository as well: https://github.com/mido/mido/issues/479#issue-1698610148. I am developing Sardine, a live-coding library for Python that uses midoas a backend for everything MIDI related. Recently, Linux users have started complaining about a recurring issues related to mido and python-rtmidi. Here is a snapshot of a typical traceback:

File "/home/ralt144mi/.local/lib/python3.10/site-packages/sardine/run.py", line 55, in <module>
    midi = MidiHandler(port_name=str(config.midi))
  File "/home/ralt144mi/.local/lib/python3.10/site-packages/sardine/handlers/midi.py", line 26, in __init__
    self._available_ports = mido.get_output_names()
  File "/home/ralt144mi/.local/lib/python3.10/site-packages/mido/backends/backend.py", line 175, in get_output_names
    devices = self._get_devices(**self._add_api(kwargs))
  File "/home/ralt144mi/.local/lib/python3.10/site-packages/mido/backends/backend.py", line 163, in _get_devices
    return self.module.get_devices(**self._add_api(kwargs))
  File "/home/ralt144mi/.local/lib/python3.10/site-packages/mido/backends/rtmidi.py", line 52, in get_devices
    mi = rtmidi.MidiIn(rtapi=rtapi)
  File "src/_rtmidi.pyx", line 817, in rtmidi._rtmidi.MidiIn.__cinit__
rtmidi._rtmidi.SystemError: MidiInAlsa::initialize: error creating ALSA sequencer client object.

The same error keeps popping up for all Linux users. We found a manual fix for it, but it is not pretty. It involves manually copying some files: /usr/lib/alsa-lib/ -> usr/lib/x86_64-linux-gnu/alsa-lib and/or usr/lib64/alsa-lib. There are variations around this command.

I know that the error is not specifically mido related, but I am wondering if someone encountered it as well and I would like to know how they were able to mitigate or fix this issue.

SpotlightKid commented 1 year ago

Sorry for the late feedback, I currently don't have much time or enthusiasm for maintenance of this project.

This error occurs when the snd_seq_open call in RtMidi fails:

https://github.com/SpotlightKid/rtmidi/blob/dd84db9a04c2d19fa744d2d4da705eaabaedcf07/RtMidi.cpp#L1813

Have you tried compiling any other ALSA example programs using this call? Do they cause fail as well?

From you workaroud solution his sounds like a Debian/Ubuntu specific problem to me. Other distros do not put libraries in /usr/lib/x86_64-linux-gnu, IIRC?

How was mido installed? From a distro package or from source? And how were the alsa libraries installed?

Bubobubobubobubo commented 1 year ago

mido is installed through Pipy as a dependency of my program. The ALSA libraries always end up in that new location on some new fresh distros. You might be right about this being a Debian specific problem as all the issues arised from Debian derivatives.

Is it possible to extend the lib scan to these new locations?

SpotlightKid commented 1 year ago

Is it possible to extend the lib scan to these new locations?

There is no "scanning" happening. The extension shared-lib links dynamically to alsa-lib in the normal fashion. It is up to the C library to find & load these libs.

SpotlightKid commented 1 year ago

So what installed the alsa libraries to /usr/lib/alsa-lib/? The debian libasound2 package does not put them there:

https://packages.debian.org/bookworm/amd64/libasound2/filelist

Bubobubobubobubo commented 1 year ago

I was never able to figure it out. It might be worth checking for other common Linux distributions that rely on their own package distribution system.

SpotlightKid commented 1 year ago

Thanks for the feedback. But I'm not sure what to do about this issue then. python-rtmidi (as any Python C/C++ extension, which links to shared system libraries) relies on the OS / the dynamic linker to be able to find and load the ALSA dynamic library correctly. It has no influence over how that works.

Bubobubobubobubo commented 1 year ago

I would be surprised if I would be the only one affected. It's probably worth keeping the issue open until someone new chimes in. I had no user complaining about the issue recently but this can change anytime soon. I'll be extra cautious and ask for more information next time the error pops up.

flokapi commented 1 year ago

I experienced the same issue on Debian11, which i could fix with: sudo ln /usr/lib/x86_64-linux-gnu/alsa-lib/libasound_module_conf_pulse.so /usr/lib64/alsa-lib/

shuckc commented 12 months ago

In my case there was a prior causing error message,

$ python miditest.py 
DEBUG:rtmidi.midiutil:Creating MidiIn object.
ALSA lib conf.c:4005:(snd_config_hooks_call) Cannot open shared library libasound_module_conf_pulse.so (/usr/lib64/alsa-lib/libasound_module_conf_pulse.so: cannot open shared object file: No such file or directory)
ALSA lib seq.c:935:(snd_seq_open_noupdate) Unknown SEQ default
Traceback (most recent call last):
  File "_rtmidi.pyx", line 819, in _rtmidi.MidiIn.__cinit__
RuntimeError: MidiInAlsa::initialize: error creating ALSA sequencer client object.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chris/repos/ola_pilot/miditest.py", line 14, in <module>
    midiin, port_name = open_midiinput(port)
  File "/home/chris/.local/lib/python3.10/site-packages/rtmidi/midiutil.py", line 272, in open_midiinput
    return open_midiport(port, "input", api, use_virtual, interactive,
  File "/home/chris/.local/lib/python3.10/site-packages/rtmidi/midiutil.py", line 203, in open_midiport
    midiobj = midiclass_(api, name=client_name)
  File "_rtmidi.pyx", line 821, in _rtmidi.MidiIn.__cinit__
_rtmidi.SystemError: MidiInAlsa::initialize: error creating ALSA sequencer client object.

Since libasound_module_conf_pulse.so is installed to /usr/lib/x86_64-linux-gnu/alsa-lib/libasound_module_conf_pulse.so (as above) I fixed with:

sudo mkdir -p /usr/lib64/alsa-lib
sudo ln /usr/lib/x86_64-linux-gnu/alsa-lib/libasound_module_conf_pulse.so /usr/lib64/alsa-lib/
SpotlightKid commented 12 months ago

Yes, all these cases point to the pulseaudio - ALSA connection being mis-configured. Whether it's by the user or Ubuntu out-of-the-box I can not say. I'm pretty sure, though, it is nothing that can be fixed by python-rtmidi.

Bubobubobubobubo commented 12 months ago

Do you know how we could relay that bug up the chain? I encountered it a few more times since I opened this issue. Last time was when installing my software on multiple Raspberry Pi (different distros) and when helping people running it on various Linux laptops.

bunnie commented 11 months ago

I also bumped into this issue on Ubuntu 22.04 and had to work around it by running

sudo ln -s /usr/lib/x86_64-linux-gnu/alsa-lib/ /usr/lib64/alsa-lib

This is on a brand new Ubuntu 22.04 installation, so this is probably Ubuntu out-of-the-box causing the issues. I'll try to file a bug report there if I can figure out their interface...

bunnie commented 11 months ago

I traced this one out a little bit...it looks like the manifest inside the ubuntu manifests do point to the right location.

However, the routine inside ALSA that is throwing the error seems to resolve the path of the shared libraries from some configuration dictionary that is passed to it. I found this note inside some of the plugins talking about the need to configure the prefix for a given system:

https://github.com/alsa-project/alsa-plugins/blob/a92944bf60f7d577c730a2856881e1ff5cbfeb88/doc/README-jack#L42-L45

This lead me to look back into the python-rtmidi source, and I see that there is a submodule of rtmidi that is complied by a wrapper script. My current speculation is that this is where the path needs to be set. A quick look through my site-installed files does show there is a hard-coded path there:

bunnie:~/.local$ grep -R "/usr/lib64"
grep: lib/python3.10/site-packages/python_rtmidi.libs/libasound-40bd9419.so.2.0.0: binary file matches

So I'm guessing somewhere in meson there needs to be a --prefix=/usr/lib/x86_64-linux-gnu argument passed to one of the build scripts. However, I have never used meson before so I'm not quite sure what the "right" way to do this -- presumably there is some variable that is provided by meson that gives you the resolution of a multi-arch path so you don't have to hard-code it inside the script.

Maybe someone with a bit more familiarity with meson can use these notes to figure out the exact patch necessary, or alternatively, I'll take any pointers as to how to do a test build with meson as a starting point to try and figure it out.

Alternatively, this could ... just be an issue with meson, I found this while googling for how to do correct multiarch support on meson: https://github.com/mesonbuild/meson/issues/1972

AL1-1956 commented 10 months ago

Hello , I am the one who made the comment about ALSA not working first time invoked but working immediately after The workaround for me , (thanks to bubobubobubobubo) is as I am on Lubuntu

"ln /usr/lib/alsa-lib/ /usr/lib/x86_64-linux-gnu/alsa-lib/libasound_module_conf_pulse.so"

If it may be useful for others ...

Excuse my poor English , because I am French and I am 67 years old

cardiffborn commented 10 months ago

Hi, I've been using python-rtmidi with mido on a rpi4 with no problems. Recently I did a clean install of Debian 12 on an imac9 and tried to run my program using Pycharm...... but it always failed in trying to find the alsa shared library.. I spent some time trying to sort out alsa issues then decided to try Mint instead..... Same issues there. However if I set the mido backend to be Pygame it's fine.

I couldn't find usr/lib/x86_64-linux-gnu/alsa-lib either.....

Just thought I'd mention it

AL1-1956 commented 10 months ago

Hello I made a mistake in the copy/paste command line What I did was

1) locate the shared library => libasound_module_conf_pulse.so

(for me on Lubuntu it is at /usr/lib/x86_64-linux-gnu/alsa-lib/)

2) create the directory /usr/lib/alsa-lib/ (wich did not exist)

3) and then made the link => ln /usr/lib/alsa-lib/ /usr/lib/x86_64-linux-gnu/alsa-lib/

so that the shared library could be found

Maybe the shared library is in another place for you , hope it can help

Alain

cardiffborn commented 10 months ago

I tried this today and it worked fine: sudo mkdir -p /usr/lib64/alsa-lib sudo ln /usr/lib/x86_64-linux-gnu/alsa-lib/libasound_module_conf_pulse.so /usr/lib64/alsa-lib/

Many thanks to everyone for helping with this.

cardiffborn commented 10 months ago

and sudo mkdir -p /usr/lib64/alsa-lib sudo ln /usr/lib/x86_64-linux-gnu/alsa-lib/libasound_module_conf_pulse.so /usr/lib64/alsa-lib/

works on Debian12

FrancescoCeruti commented 8 months ago

We're experiencing the same issue here.

From what I can see, python-rtmidi is not directly responsible. As @bunnie commented, the error originates from [...]/python_rtmidi.libs/libasound-40bd9419.so.2.0.0.

This file is likely copied from the 'build machine' by cibuildwheel during the build process, but it seems to be not fully compatible with all distributions.

Is this the intended behavior, @SpotlightKid?

SpotlightKid commented 8 months ago

Ok, I think I'm starting to get a picture of what is happening here:

  1. The binary wheels of python-rtmidi for Linux contain their own copies of libasaound and libjack in python_rtmidi.libs/libjack-5305d520.so.0.1.0 and python_rtmidi.libs/libasound-40bd9419.so.2.0.0. I suppose they are put there by cibuildwheel / auditwheel.

    These get installed in <venv>/lib/python3.11/site-packages/python_rtmidi.libs when you install python-rtmidi from those wheels into a virtual environment.

  2. The python-rtmidi C extension shared library links directly to those two shared library files using a relative path. If you remove these shared lib files, the python-rtmidi C extension breaks, i.e. it won't look for the system-wide asound or jack libs.

    So, I have to retract my statement above, that finding the linked libs is always governed by the dynamic linker configuration. TBH, I didn't know you could do this.

  3. The included libasound shared lib seems to contain a hard-coded path for where to look for ALSA plugins (also shared libraries) and this is set to /usr/lib64/alsa-lib. This doesn't match the path where these ALSA plugins are actually installed on debian and derived distros.

    Hence, when the ALSA configuration of the system makes it necessary to load an ALSA plugin, and when the python-rtmidi C extension tries to load libasound and this in turn tries to load an ALSA plugin library, this fails.

I have to do some research to find out how to solve this, e.g. either by telling cibuildwheel not to include libasound and libjack in the wheel or by setting the ALSA plugin library path in some other way (changing it at compile is not an option, since the point of the binary wheel built with cibuildwheel is to have a wheel, which works across distros).

SpotlightKid commented 8 months ago

Alright, I've changed the CI, so that the Linux x86_64 / aarch64 binary wheels no longer include libasound*.so.*. Can someone, who had the problem reported above, please try one of these binary wheels from the test builds?

  1. Download the CI artifact: https://github.com/SpotlightKid/python-rtmidi/suites/18310201368/artifacts/1059196804

  2. Unzip the file artifact.zip

  3. Create a virtual env and activate it:

    python -m venv pmvenv
    source pmvenv/bin/activate
  4. install the wheel matching your Python version, for example for Python 3.11:

    python -m pip install artifact/python_rtmidi-1.5.7-cp311-cp311-manylinux_2_28_x86_64.whl
  5. Do a smoke test:

    python
    >>> import rtmidi
    >>> mi = rtmidi.MidiIn()
    >>> print(mi.get_ports())

You can simply close the terminal and delete the pmvenv directory and the artifacts when you're done.

bunnie commented 8 months ago

Thanks for doing this. If nobody else gets around to it, I'll give it a try in about a week when I'm reunited with my midi controller again.

FrancescoCeruti commented 8 months ago

Thanks! I've tried on mint-21.2 (based on ubuntu-22.04), the error is gone, and the lib is using the system libasound.

Here's the output of objdump -p _rtmidi.cpython-310-x86_64-linux-gnu.so

  RPATH                $ORIGIN/../python_rtmidi.
  NEEDED               libasound.so.2
  NEEDED               libjack-5305d520.so.0.1.0
  NEEDED               libstdc++.so.6
  NEEDED               libgcc_s.so.1
  NEEDED               libpthread.so.0
  NEEDED               libc.so.6
eliasdorneles commented 8 months ago

I have also just tried the wheel version from the CI artifact, and verified that it works on a virtualenv:

$ pip install ./python_rtmidi-1.5.7-cp310-cp310-manylinux_2_28_x86_64.whl 
Processing ./python_rtmidi-1.5.7-cp310-cp310-manylinux_2_28_x86_64.whl
Installing collected packages: python-rtmidi
Successfully installed python-rtmidi-1.5.7
$ python
Python 3.10.12 (main, Jun 11 2023, 05:26:28) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import rtmidi
>>> rtmidi.MidiOut()
<_rtmidi.MidiOut object at 0x7fdb562e2ed0>
>>> rtmidi.MidiIn()
<_rtmidi.MidiIn object at 0x7fdb562e35b0>
>>> 
bunnie commented 7 months ago

I know the issue is already closed, but just adding that it also worked on my platform as well. Thank you!