Feature Request: Support files mounted via `mpremote mount`

[IanLondon]

Hello, thanks for this library!

The way umidiparser reads files seems to be incompatible with mpremote mount.

Setup

• Raspberry Pi Pico
• mpremote 1.22.0
• Running on Linux Mint

Steps to reproduce

1. Install to USB-connected Pi Pico via: mpremote mip install github:bixb922/umidiparser

2. Save a MIDI file eg in local src/my_midi.mid

3. Make a script like this in local src/app.py:

import umidiparser
import utime
for event in umidiparser.MidiFile("my_midi.mid").play():
    print(event )

3. Use mpremote mount : mpremote mount src exec "import app" to mount and run your script

Result:

File "/lib/umidiparser/umidiparser.py", line 958, in __init__
AttributeError: 'RemoteFile' object has no attribute 'tell'

Workaround

If you don't use mount and just fs cp the files to your board, it works fine.

Guessing why

As far as I can tell, mpremote is not exposing the full file API in its RemoteFile object.

https://github.com/micropython/micropython/blob/5114f2c1ea7c05fc7ab920299967595cfc5307de/tools/mpremote/mpremote/transport_serial.py#L726

If that's correct, then this is really a Micropython issue, but it might be possible for this library to support RemoteFile and therefore mount.

[bixb922]

Yes, your analysis is correct. What umidiparser does is the following. The tracks in the file have to be merged, so umidiparser opens a file for each track, reads the tracks piecemeal to a small buffer to lower RAM requirements, and does the merge on the fly. However, to remember where each track starts, seek() and tell() operations are needed, and as you point out, these are not available for mpremote mount.

A workaround is opening the MIDI file with buffer_size=0, for example: for event in umidiparser.MidiFile("my file.mid", buffer_size=0):

This will avoid using tell() and seek() functions on the file. The complete midi file will be read into memory in one go.

I verified this on a ESP32 and a RP2040 doing mpremote mount . run test_program.py and it worked well. However, it may be advisable to put umidiparser.py (or better: cross-compiled to umidiparser.mpy) in the /lib folder of the microprocessor, because loading the umidiparser.py module with mpremote mount can be a bit slow and memory intensive.

Please tell me if this worked for you.

[bixb922]

The downside of buffer_size=0 is that it uses more RAM, since the whole file has to be buffered in RAM. Even if there is enough RAM available, it is likely that there is not enough contiguous RAM to buffer the file. This will trigger the following error: MemoryError: memory allocation failed, allocating 20755 bytes, where the number is the file size.

I can think of a workaround that does not raise memory requirements, but this workaround would slow down opening MIDI files considerably, since I would replace seek() with reading the parts to be skipped. It would be quite fast, however for MIDI type 0 files (single track MIDI files). Any type 1 MIDI file can be transformed to a MIDI type 0, there are small Python programs on internet doing that, I have done that too with the Python Mido library.

All this would only affect files accessed on the mpremote mount virtual file system.

Please tell me if you think this is approach useful and necessary for your case.

[bixb922]

Just in case it's useful: there is still another option: when using mpremote mount, the flash is still visible at /. So if you have a /midi_files folder or a /lib folder on flash, you can access these using the absolute path with the initial /, i.e. MidiFile("/midi_files/myfile.mid")

While mpremote mount is active, the files on the PC appear in the virtual /remote folder, but since mpremote does an automatic cd to /remote, they appear in the current folder. That way the files on the PC can be opened with a relative path (without / at the start), but the files on flash are still at / and subfolders.

Later in the development cycle, when not using mpremote mount anymore, that transition can be made totally transparent to the program, just using absolute and relative file and folder names.

[bixb922]

I updated the README with instructions about this restriction. I believe the workarounds described should be enough for most use cases (but I can be wrong there). Getting rid of tell()/seek() comes at a large processing overhead, except perhaps for MIDI type 0 files, making that change unattractive for microcontrollers such as the RP2040 or ESP32. On big Python on a PC, one could just convert each track to a list, append those lists and then sort that big list. That approach is, of course, unfeasible for a microcontroller, because this consumes a lot of RAM. Hence using tell()/seek() and merging tracks on the fly really makes parsing MIDI files feasible with the RAM available on most microcontrollers.

Please comment if you have any question or concern.