Feature request - Lights on midi playback

[vzoltan]

Hi, id like to send a feature request.

It would be cool if the lights would light up during midi playback, to "simulate" the feature of those old pianos that played the song and also the keys.

Thanks

[vzoltan]

btw, if it will be added, it also can used as a synthesia alternative, youd be able to play on top realtime. like free play, but with leds on to show what to play.

[onlaj]

Added, but I don't think there is any use for that. Even when I launch example script playback is slower by 2-8%, depending on how advanced the midi file is. I think Zero is too slow to handle it without drift, at least not in Python.

[vzoltan]

when i play the midi file with this https://github.com/vzoltan/Piano-LED-Visualizer/blob/master/playmidi.py

and restart the visualizer while that script loads the midi,

i have no delay between the lights and the audio.

[vzoltan]

https://photos.app.goo.gl/NNhHP13RLXDKt62Y8

[vzoltan]

i dont know if it has anything to do with it, but i run the visualizer as a service: https://github.com/vzoltan/Piano-LED-Visualizer/blob/master/system_files/lib/systemd/system/pianoled.service and i restart that service:

so i play the midi like this:

https://github.com/vzoltan/Piano-LED-Visualizer/blob/master/system_files/var/www/html/pm.sh

[onlaj]

No visual delay, right. But compare how long midi file is played to the "real" time. Usually difference is about 5%.

[vzoltan]

i dont understand. the playback the lights everytihng is in real time, or real speed. it stays in sync till the end. could you pls send me your midi file, id like to try it with my setup like this.

[onlaj]

It stays in sync, but midi is played 2-8% slower than it should (at least on my setup). I can clearly hear that, don't even need to measure anything.

[vzoltan]

pls, share that midi, id like to try it on mine later tonight. would you?

[onlaj]

I tried ~20 midi files, doesn't matter.

This is my code for measures:

#!/usr/bin/env python

import sys
import mido
import time
from mido import MidiFile

portname = ""
filename = ""
with mido.open_output(portname) as output:
    try:
        midifile = MidiFile(filename)
        t0 = False
        length = midifile.length
        for message in midifile.play():
            if(t0 == False):
                t0 = time.time()
            output.send(message)
        print('play time: {:.2f} s (expected {:.2f})'.format(
                time.time() - t0, length))

    except KeyboardInterrupt:
        print()
        output.reset()

Fill empty spaces for portname and filename with your port and midi path.

[vzoltan]

ok, ill try in 2 hrs.

[vzoltan]

now i see your point.

https://youtu.be/GiZDgdE8SUk?t=30 she plays the part from 0:31 to 0:59

https://photos.app.goo.gl/NNhHP13RLXDKt62Y8 where my video same part: 0:06 to 0:40

28 vs 34 sec (if im right) :/ thats a lot difference and this is her midi file :/

[onlaj]

Yep. I think playing midi with something written in c++ or anything faster than Python would work. Maybe I will add it in future. Or maybe it is just mido library doing it wrong. I think it doesn't take into an account that outputting message to port also takes a time.

[vzoltan]

if you find any script or software that already exist and could send midi to the piano on linux from shell (promt) pls let me know :)

[onlaj]

I made some tweaks. Before: (Chopin - Revolutionary Etude op.10 no.12)

play time: 151.95 s (expected 138.81)

After:

play time: 143.25 s (expected 138.81)

After few more tweaks:

play time: 140.06 s (expected 138.81)

Different midi just to make sure: (Liszt - La Campanella)

play time: 244.26 s (expected 242.46)

And something slower: (Yann Tiersen - Comptine d`un autre ete)

play time: 129.39 s (expected 128.99)

Not perfect, but difference is indistinguishable, at least not for my ears.

[vzoltan]

Could you share the code pls? Id like to try it :) (But only tomorrow)

[onlaj]

I think it is possible to improve it but I'm happy with the result so far, less than 1% difference with really fast midis.

import sys
import mido
import time
from mido import MidiFile

portname = ""
filename = ""

output_time_last = 0
delay_debt = 0;
with mido.open_output(portname) as output:

    try:
        midifile = MidiFile(filename)
        t0 = False
        length = midifile.length        

        for message in midifile:           
            if(t0 == False):
                t0 = time.time()
                output_time_start = time.time()            
            output_time_last = time.time() - output_time_start            
            delay = message.time - output_time_last + delay_debt

            if(delay > 0):
                time.sleep(delay)
                delay_debt = 0
            else:
                delay_debt += message.time - output_time_last

            output_time_start = time.time()                   

            if not message.is_meta:
                output.send(message)

        print('play time: {:.2f} s (expected {:.2f})'.format(
                time.time() - t0, length))

    except KeyboardInterrupt:
        print()
        output.reset()

[vzoltan]

before: play time: 325.74 s (expected 310.53)

after: play time: 312.64 s (expected 310.53)

did you try to restart during the visualizer, you also get the leds on, right?

[vzoltan]

filename = sys.argv[1]

like this it can play any midi, so i can use it now with my "web player" ;)

THX

btw, second play was: play time: 312.53 s (expected 310.53) a bit even better :)

[onlaj]

did you try to restart during the visualizer, you also get the leds on, right?

Restart what? When I launch playmidi.py with visualizer.py running at the same time it doesn't light up keys.

As I suspected default mido's "play()" method doesn't care that output.send also takes time to execute. So if there is 0.001 second delay between notes and it took 0.001 second to send previous note it means the total delay will be 0.002. Multiply it by 4000 notes and you get 4 seconds delay on one song. Modified code subtracts execution time from delay. Something is still slowing it down but I can't figure what exactly.

[vzoltan]

by restart i mean: you start your midi file with this python you posted here, and open a new terminal window and restart visualizer while that python runs, you'll get your leds following the music.

run your midi via that python after you hit enter, you can already restart you visualizer in the background (or new window), no need to wait for any sound/note from the piano.

[onlaj]

Nope, doesn't work like that for me and I have no idea how it is a thing for you. playmidi.py sends messages to output, it should never go back to input. Seems like your piano is sending messages back?

[vzoltan]

Ok, here is what i do from zero:

tur on piano, turn on pi. (set nothing on pi) start the midi with your script restart visualizer leds on.

video: (there is a cut, but only the midi loading time, which is a lot) https://photos.app.goo.gl/ttwE6YRBco4LbUu29

[onlaj]

Doesn't work for me.

Anyway, I made some more tweaks and tested more midis, this is how it looks now.

play time: 140.75 s (expected 138.81)
play time: 140.93 s (expected 138.81)
play time: 244.32 s (expected 242.46)
play time: 216.27 s (expected 214.61)
play time: 272.44 s (expected 271.67)
play time: 177.28 s (expected 176.06)
play time: 239.56 s (expected 236.41)
play time: 78.37 s (expected 77.62)
play time: 142.55 s (expected 141.29)
play time: 192.96 s (expected 192.00)
play time: 224.38 s (expected 223.80)
play time: 195.55 s (expected 194.65)
play time: 272.46 s (expected 271.67)
play time: 264.32 s (expected 261.33)
play time: 285.80 s (expected 283.97)

And just for fun some black midi: play time: 294.93 s (expected 212.62)

[vzoltan]

Where can i find the tweaks?:)

Also what is black midi?

If it doesnt work for you, how are you going to make the leds follow the midi?

[onlaj]

I will commit tomorrow, after some more tests. https://en.wikipedia.org/wiki/Black_MIDI https://www.youtube.com/watch?v=MueyK8YSvQM

It works with visualizer.py because I inject midi messages into input port.

[vzoltan]

So then you can make it to have 2 colors by the two keys?(thats not perfect ni know but better than just one color;) ) just for eye-candy

Since all this enhancement is just for eye-candy:)

Black midi:) i would not play midi on the piano with more instruments, just one, the piano :)

[vzoltan]

it works with the default ligth settings.

although you are able to load midis multiple times, and multiple midis at the same time bc there is no feedback while the midi loads, it gives back the midi list immediately.

also, could ypu pls share the even more tweaked midi playback python script, to let me use it on my "web player" please?

Thanks

[onlaj]

although you are able to load midis multiple times, and multiple midis at the same time bc

Yep, I will make sure that only one midi can be played at the same time in next update.

also, could ypu pls share the even more tweaked midi playback python script, to let me use it on my "web player" please?

You can see changes in latest visualizer.py. I just added fixed time compensation (3 milliseconds works just fine).

[vzoltan]

just this: delay = message.time - output_time_last - float(0.003) + delay_debt

?

[onlaj]

yes

[vzoltan]

play time: 291.18 s (expected 310.53) ;)

btw do you plan on add a feature like blue and green lights on playback? (just by the treble clef and bass clef)

[onlaj]

play time: 291.18 s (expected 310.53)

So I guess if you run it as a separate script you don't need additional compensation. You can try to lower it to 1ms or something.

Yeah, I can add it as a optional feature.

[vzoltan]

play time: 291.18 s (expected 310.53) So I guess if you run it as a separate script you don't need additional compensation. > You can try to lower it to 1ms or something.

THX

Yeah, I can add it as a optional feature.

Great!:)

[vzoltan]

0.001 play time: 305.35 s (expected 310.53) without that extra 0.001: play time: 313.08 s (expected 310.53)

[vzoltan]

Hi, i saw this enhancement is still open, so i thought im gonna add to it an "extension" :)

so, not that it works perfectly, could you add a feature/settings in the menu, where it would be possible to set "right and left hand" different color by the Treble and Bass cleff, like in synthesia? just for playback as eye-candy, and if its gonna be implemented, also differentiate the black key and the white key with deeper blue/green color, just the way it is in synthesia.

just a nice to have feature request :) or should i create a new one for this? (so this is not for the finger-based channel in synthesia, there is a feature request for that already, its just for simple midi playback without pc or tablet)