Python 3.9+ support (Rtmidi-python)

[theredled]

Required rtmidi-python is a 2014 library. Therefore, it's not working anymore with Python >= 3.9 (https://stackoverflow.com/questions/71410788/pip-failed-building-wheel-for-rtmidi-python-subprocess-exited-with-error-and-le).

Problem is current Python version is 3.12, and even Raspberry Pi OS legacy version (11 Bullseye) is bundled with Python 3.9.

Any idea to get around this?

[josephernest]

Hi @theredled, thanks for the report! (which has been reported a few times)

I agree this rtmidi-python might be a problem in the future if not supported anymore, but if we change this library by a replacement, we should first confirm that it works:

1. for Py 3.7-3.8 on RPi (many people still use this on currently working sytems)
2. for Py 3.9+ on RPi
3. for Py 3.7-3.8 on Windows
4. for Py 3.9+ on Windows
5. it allows multiple MIDI devices connected at the same time (the current solution allows this)
6. it supports hot-plugging of new MIDI devices on-the-fly, without restarting the Python script
7. it uses a MIDI library which is easily installable (pip install ...) on both RPi and Windows (without requiring the user to install VC++ for the Windows case)

If you are interested, could you provide a demo test.py demonstrating access to MIDI devices, that fits all these requirements?

It would be super interesting to have this!

[theredled]

Thanks for your answer! I'm quite new to all of this, but for what I've been implementing in the last few weeks (incl. communication between several midi devices), I know that mido supports connection of multiple MIDI devices, and hot-plugging (as long as you're watching, as your script do), and I think it's based on python-rtmidi. I'll try to make a script, shouldn't be hard, and see if it meets the other requirements – if it didn't, it would be useful information anyway.

My personal purpose would be to put it on an existing RPi beside other applications. Maybe it's doable by having several python versions installed – but as you seem open to it, it's probably better to do sth about that rtmidi-python "weakness".

[josephernest]

Thanks @theredled! The main challenge before switching from "rtmidi-python" to any other new library is that each condition 1. 2. 3. 4. 5. 6. 7. is still preserved. Very often I'm used to the fact that the devil is in the details :)

[theredled]

Side note, mido is actually an abstraction layer above a backend, which is python-rtmidi by default, but it can also be rtmidi-python. Maybe it can be an extra lead https://mido.readthedocs.io/en/stable/backends/index.html

[theredled]

Doesn't seem to buildable on windows without VC++ : https://spotlightkid.github.io/python-rtmidi/install-windows.html I guess the reason of that requirement is you're working on windows? But why not install VC++? Do many people use SamplerBox on windows (+ with an old python + without VC++)?

If it remains really important to you, do you mind if I do a fork with updated dependencies and without that Windows requirement? Thanks!

[josephernest]

@theredled Thanks for your feedback.

Yes some people use SamplerBox on Windows. (I personally work on both Linux and Windows). Also some people still use SamplerBox with RPi2 and not-latest versions of Raspbian/RPiOS (so they don't have Py 3.12, but rather 3.7 or 3.8).

I mistyped my previous message, here is the correct version: "easily buildableinstallable on Windows without VC++".

I mean someone without VC++ installed should be able to pip install the-midi-library and it should install on Windows. Reason: if the end-user on Windows has just to pip install ..., and run python samplerbox.py, it's still doable for someone who is not an experienced developer.
If he/she has to install VC++, ensure that everything works well in the build process, it's much more complicated, and nearly impossible for a non-dev person.

a fork with updated dependencies and without that Windows requirement?

I don't think it's a good idea to drop a platform, I built everything to be as cross-platform as possible ;)

Can you provide a small test.py with python-rtmidi that lets us test that conditions 1. to 7. are ok on multiple platforms (I can do the tests on Win, and you do the other tests?)? Then we could merge it into main project, it would be better for everyone :)

[theredled]

Hi @josephernest!

Do we agree that:

• Whatever decision we (you) make, people will always be able to use current version of Sampler Box (let's call it 1.0), on Windows, old Python, without C++
• For the moment, it is not possible to install Samplerbox on recent (2020+) and future software

Extra question, why are non-dev people using samplerbox on windows? Isn't designed to be a RPi software?

Why wouldn't we release current version (1.0) and then make a new release (1.1, 2.0 ?) that trade "Compatible with Windows for non-devs" requirement for "Compatible with recent software"?

[theredled]

As for test.py I wrote this, working on macos and RPi with Python 3.9+ but I know it won't be working on windows without VC++. Waiting for your answer to last comment to test with older Pythons (not that easy to do)

import rtmidi, time

def MidiCallback(message, time_stamp):
    print(message)

midi_in = [rtmidi.MidiIn()]
previous = []
while True:
    all_ports = midi_in[0].get_ports()
    for port_num,port in enumerate(all_ports):
        if port not in previous and 'Midi Through' not in port:
            print('new ports found in : ', all_ports)
            midi_in.append(rtmidi.MidiIn())
            midi_in[-1].set_callback(MidiCallback)
            midi_in[-1].open_port(port_num)
            print('midi_in', midi_in)
            print('Opened MIDI: ' + str(port))
    previous = midi_in[0].get_ports()
    time.sleep(2)

[Hurtknight]

I agree this rtmidi-python might be a problem in the future if not supported anymore, but if we change this library by a replacement, we should first confirm that it works:

for Py 3.7-3.8 on RPi (many people still use this on currently working sytems) for Py 3.9+ on RPi for Py 3.7-3.8 on Windows for Py 3.9+ on Windows it allows multiple MIDI devices connected at the same time (the current solution allows this) it supports hot-plugging of new MIDI devices on-the-fly, without restarting the Python script it uses a MIDI library which is easily installable (pip install ...) on both RPi and Windows (without requiring the user to install VC++ for the Windows case) If you are interested, could you provide a demo test.py demonstrating access to MIDI devices, that fits all these requirements?

It would be super interesting to have this!

[theredled]

I agree this rtmidi-python might be a problem in the future if not supported anymore, but if we change this library by a replacement, we should first confirm that it works:

for Py 3.7-3.8 on rRPi (….

Why did you c/p 2nd post ?

[josephernest]

@theredled I did not, @Hurtknight seems to be a spam bot.

I haven't had time to answer your other messages, I'll do it in the next days.

[theredled]

No pb!

[theredled]

For the record, I forked a Python 3.9-compatible and a bit rewritten/refactored version : https://github.com/theredled/SamplerBox No support for loops at the moment (some problem with chunk library)

[dosas]

As far as I can see rtmidi-python now supports all current python versions?

[theredled]

As far as I can see rtmidi-python now supports all current python versions?

That's python-rtmidi, not rtmidi-python, different librarires.

[dosas]

As far as I can see rtmidi-python now supports all current python versions?

That's python-rtmidi, not rtmidi-python, different librarires.

Argh ... who chooses names like this ;)

[josephernest]

@dosas The main challenge before switching from "rtmidi-python" to any other new library is that these conditions are preserved, in particular:

5. it allows multiple MIDI devices connected at the same time (the current solution allows this)
6. it supports hot-plugging of new MIDI devices on-the-fly, without restarting the Python script

Did someone test this thoroughly with python-rtmidi?

These were very important design conditions at the start of the project (many people are using multiple MIDI devices at the same time and/or hot-plugging of new MIDI devices on the fly. Example: you start SamplerBox, then you plug a new USB MIDI keyboard => it should be recognized. This is guaranteed with rtmidi-python ; is it the case with other libraries?).

[dosas]

@dosas The main challenge before switching from "rtmidi-python" to any other new library is that these conditions are preserved, in particular:

5. it allows multiple MIDI devices connected at the same time (the current solution allows this)

6. it supports hot-plugging of new MIDI devices on-the-fly, without restarting the Python script

Did someone test this thoroughly with python-rtmidi?

I cannot test it at the moment because I have no midi keyboard/controller available.

@josephernest I would assume as maintainer/owner of the project you probably have some different raspberry pi models and maybe n+1 MIDI devices? Furthermore you have the domain knowledge.

Would you mind installing the code from the PR and testing items 5 and 6?

These were very important design conditions at the start of the project (many people are using multiple MIDI devices at the same time and/or hot-plugging of new MIDI devices on the fly. Example: you start SamplerBox, then you plug a new USB MIDI keyboard => it should be recognized. This is guaranteed with rtmidi-python ; is it the case with other libraries?).

I get that you want to keep the features as is but what is the exit strategy here?

Have a deprecated and continuously less usable project with the full feature set or have an up to date and usable project with a slightly reduced feature set?

With this PR the feature set is kept as is for the 'old devices' (depending on the python version) and the 'new devices' will work with a possible slightly reduced feature set.

[josephernest]

After analysis, I notice superquadratic/rtmidi-python works very well with Python 3.9+.

See solution here: https://github.com/superquadratic/rtmidi-python/issues/25

Therefore no need to change the MIDI library for SamplerBox. What I've learnt from decades of programming is that changing one dependency (that worked perfectly during 10 years) to a very-similar-one often brings surprises, and other unexpected things, that are complicated to solve. Therefore, since rtmidi-python works on Python 3.9+, no need to risk this (and discover later, for example, that using multiple MIDI devices + hotplugging doesn't work perfectly anymore, or similar things...).

I will update the installation instructions for Python 3.9+ soon.

[josephernest]

Solution:

pip3 install git+https://github.com/SamplerBox/rtmidi-python.git

Thank you @theredled @dosas @mojca for having reported this.

[mojca]

That does not seem to work for me.

(venv) D:\temp\python>pip install cython
Collecting cython
  Downloading Cython-3.0.10-cp311-cp311-win_amd64.whl.metadata (3.2 kB)
Downloading Cython-3.0.10-cp311-cp311-win_amd64.whl (2.8 MB)
   ---------------------------------------- 2.8/2.8 MB 2.9 MB/s eta 0:00:00
Installing collected packages: cython
Successfully installed cython-3.0.10

(venv) D:\temp\python>pip install git+https://github.com/SamplerBox/rtmidi-python.git
Collecting git+https://github.com/SamplerBox/rtmidi-python.git
  Cloning https://github.com/SamplerBox/rtmidi-python.git to c:\users\me\appdata\local\temp\pip-req-build-8o4jgr9h
  Running command git clone --filter=blob:none --quiet https://github.com/SamplerBox/rtmidi-python.git 'c:\users\me\AppData\Local\Temp\pip-req-build-8o4jgr9h'
  Resolved https://github.com/SamplerBox/rtmidi-python.git to commit 4eb4ebb0c7b0a7a81957507051c65a346f170529
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error

  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [20 lines of output]
      Traceback (most recent call last):
        File "D:\temp\python\venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
          main()
        File "D:\temp\python\venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "D:\temp\python\venv\Lib\site-packages\pip\_vendor\pyproject_hooks\_in_process\_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "c:\users\me\AppData\Local\Temp\pip-build-env-8ylnibg7\overlay\Lib\site-packages\setuptools\build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "c:\users\me\AppData\Local\Temp\pip-build-env-8ylnibg7\overlay\Lib\site-packages\setuptools\build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "c:\users\me\AppData\Local\Temp\pip-build-env-8ylnibg7\overlay\Lib\site-packages\setuptools\build_meta.py", line 487, in run_setup
          super().run_setup(setup_script=setup_script)
        File "c:\users\me\AppData\Local\Temp\pip-build-env-8ylnibg7\overlay\Lib\site-packages\setuptools\build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 4, in <module>
      ModuleNotFoundError: No module named 'Cython'
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

[josephernest]

@mojca The installation instructions, and SamplerBox support in general, have always been for x86-64 Linux and RaspiOS (arm).

For Windows, but you're a bit more on your own (as it has always been) :wink:

Now the solution, tested right now with Windows 10, Python 3.11, Cython 3.0.10 : instead of the pip install git+https..., just do (see also this SO post)

pip install cython
git clone https://github.com/SamplerBox/rtmidi-python.git
cd rtmidi-python
python setup.py install

I just updated the documentation accordingly.

Happy that it finally works for Linux, RaspiOS, and Windows! :tada:

[dosas]

@dosas The main challenge before switching from "rtmidi-python" to any other new library is that these conditions are preserved, in particular:

5. it allows multiple MIDI devices connected at the same time (the current solution allows this)

6. it supports hot-plugging of new MIDI devices on-the-fly, without restarting the Python script

Did someone test this thoroughly with python-rtmidi?

These were very important design conditions at the start of the project (many people are using multiple MIDI devices at the same time and/or hot-plugging of new MIDI devices on the fly. Example: you start SamplerBox, then you plug a new USB MIDI keyboard => it should be recognized. This is guaranteed with rtmidi-python ; is it the case with other libraries?).

BTW: I tested it with python-rtmidi: Hot plugging works, as well as multiple midi devices, and midi via serial