Feature request: Note off event

[peterkorgaard]

I'm using MidiPlayerJS together with soundfont-player. This works in most cases fine, but sometimes it is really audibly, that it do not send note off events.

[grimmdude]

Hi @peterkorgaard,

Thanks for your message. I believe the Note Off events might be omitted if a MIDI file is using running status. Could you attach the MIDI file you're using and I will make necessary changes to the player. Thanks,

-Garrett

[peterkorgaard]

Hi @grimmdude Thanks for getting back on this.

I do not know what running status is, but what I can tell you is, that I'm using Verovio to generate scores and midi for that particular score. What I have noticed is, that even though the score contains pauses, the midiplayer do not parse those. I have made an example score to demonstrate.

This consists of notes stopped by 3 pauses. What I really would like is that every note sends a note on when it begins and a note off when it ends. I think this information must be within the midi file. The generated base64 encoded midi looks like below.

data:audio/midi;base64,TVRoZAAAAAYAAQACAHhNVHJrAAAABAD/LwBNVHJrAAAANACQPEB4kDwAeJA+QHiQPgB4kEBAPJBAAACQQEA8kEAAPJBAQDyQQAAAkDxAeJA8AAD/LwA=

The problem when note off is not send is, that it plays back like this:

I hope this explains, what I mean. I do not have the binary version of the midi file. Only the base64 encoded version, but any midi file containing pauses could be used.

[peterkorgaard]

Found a way to decode the base64 encoded midi:

4d54 6864 0000 0006 0001 0002 0078 4d54 726b 0000 0004 00ff 2f00 4d54 726b 0000 0034 0090 3c40 7890 3c00 7890 3e40 7890 3e00 7890 4040 3c90 4000 0090 4040 3c90 4000 3c90 4040 3c90 4000 0090 3c40 7890 3c00 00ff 2f00

And a way to convert this into readable JSON { "header": { "PPQ": 120, "bpm": 120, "name": "" }, "startTime": 0, "duration": 3.5, "tracks": [ { "startTime": 0, "duration": 0, "length": 0, "notes": [], "controlChanges": {}, "id": 0 }, { "startTime": 0, "duration": 3.5, "length": 6, "notes": [ { "name": "C4", "midi": 60, "time": 0, "velocity": 0.5039370078740157, "duration": 0.5 }, { "name": "D4", "midi": 62, "time": 1, "velocity": 0.5039370078740157, "duration": 0.5 }, { "name": "E4", "midi": 64, "time": 2, "velocity": 0.5039370078740157, "duration": 0.25 }, { "name": "E4", "midi": 64, "time": 2.25, "velocity": 0.5039370078740157, "duration": 0.25 }, { "name": "E4", "midi": 64, "time": 2.75, "velocity": 0.5039370078740157, "duration": 0.25 }, { "name": "C4", "midi": 60, "time": 3, "velocity": 0.5039370078740157, "duration": 0.5 } ], "controlChanges": {}, "id": 1, "channelNumber": 0, "isPercussion": false } ] }

[peterkorgaard]

Hi Garret

I think the issue with the note off event can be solved simply by sending the duration parameter along with the event object. Right now I can see the following parameters passed with note on.

The soundfont-player is having the abillity to secure the propper length of a note when played. Together with MidiPlayerJS the use could be something like below:

[grimmdude]

Hi @peterkorgaard,

So, the duration parameter is already being sent in the event object within the delta property. That number is how many ticks the event should last for. Does that satisfy your need?

This is incorrect, see https://github.com/grimmdude/MidiPlayerJS/issues/5#issuecomment-337715899

-Garrett

[grimmdude]

The reason the Note Off event isn't being emitted here is because it doesn't exist in the MIDI file you attached. Omitting this event is common in MIDI because it reduces the number of bytes needing to be processed thus the playback has less latency and the file size is smaller. Using this method is called "running status".

So the thing is, if I made a change in this library to send the "Note Off" event when running status is used it would basically defeat the purpose of using running status in the file.

-Garrett

[peterkorgaard]

You are right, Garret. It is not the missing note off event that gives the issue. The problem, I think is in the duration of the tone played. I have examined the events passed playing this:

The peculiar thing is, that for every note on event with a velocity greater than 0 an identical event with velocity 0 is send.

First i thought I could use this to stop the note, but this do not function because for every event a new Midiplayer.player is created. So every event is on it's own. For that reason I need to know the duration of the tone when it is played to tell soundfont-player when to stop the tone again.

Looking at the delta event I do not think it always tells the duration of the tone to play. First event in the above example has a delta of 0 and is played on tick 0.

I think the duration is the difference between note on and the time, when the note is intended to stop playing - the note off not send.

[peterkorgaard]

The delta do have the duration in some events but not the frist and the second last. These should both have a delta on 120.

[grimmdude]

Hi @peterkorgaard,

In running status I believe those events with 0 velocity and 0 delta indicate off trigger for the previous event. Check out this article for more info: https://www.midikits.net/midi_analyser/running_status.htm

-Garrett

[grimmdude]

Also, I don't believe a new MidiPlayer.player should be created for each event. Is this intentional in your code?

[peterkorgaard]

Hi Garrett

Thanks for the article. Now I understand running status better

I just misunderstood the code. I'm actually using just one new Midiplayer.player. My fault.

I'm trying to use the 0 velocity as an indicator for when a particular note needs to be stopped. This seems to do the trick- except that soundfont-player only does it sometimes. So so far so good.

I still think it is peculiar the first event with a velocity greater than 0 has a delta on 0. The same goes for the last event - before end of track. Are there an explanation for this?

[grimmdude]

Hi @peterkorgaard,

Great, glad those events will work for you. I'm not sure why those first/last events has 0 delta/velocity, this comes directly from the MIDI file so it must have been generated by the software which was used to create the file. They could just be there to pad the live running status events.

-Garrett

[peterkorgaard]

Hi Garrett

I do not think the delta value 0 is from the midi file. If I use https://tonejs.github.io/MidiConvert/ to pass the midi as JSON the first event is: { "name": "C4", "midi": 60, "time": 0, "velocity": 0.5039370078740157, "duration": 0.5 }

And the last event is: { "name": "C4", "midi": 60, "time": 3, "velocity": 0.5039370078740157, "duration": 0.5 }

The duration is somehow shown in seconds, but if it was based on the delta value and this was 0 then the duration should have been 0 as well, i think.

[grimmdude]

Hi @peterkorgaard,

MidiPlayerJS just outputs the events that it reads in the MIDI file. I see this initial event with 0 value for delta in the source of the file (see bold values below). My guess is that MidiConvert library you're using to get the JSON is omitting this event for whatever reason.

4d54 6864 0000 0006 0001 0002 0078 4d54 726b 0000 0004 00ff 2f00 4d54 726b 0000 0034 0090 3c40 7890 3c00 7890 3e40 7890 3e00 7890 4040 3c90 4000 0090 4040 3c90 4000 3c90 4040 3c90 4000 0090 3c40 7890 3c00 00ff 2f00

I'm also noticing that this file contains an "End of Track" event near the beginning of the file which is strange (00FF 2f00). What did you use to generate this file?

-Garrett

[peterkorgaard]

Hi Garrett

I'm in process of creating music theory education. To do this I use Verovio (https://github.com/rism-ch/verovio or http://www.verovio.org/index.xhtml) to create the scores (svg). The scores has the option to also render to midi. The input to the score is from either a string format called PAE or something called MEI which is an xml file. This all works fine - at least in relation to the score rendering. To play back the midi i first used midi.js, but when I saw your demo of MidiPlayerJS (http://grimmdude.com/MidiPlayerJS/), I decided to change to your software together with soundfont-player. It is much more light and plays back more fluently :-).

Doing that I realized that it was not enough to just to play back noteOn events. To play back accurately the duration of the notes needed to be taken into account. This was when I wrote to you. The midi file I have shown is just a simple example with pauses I have made using Verovio for testing.

Examining the midi created I can see the End of track event in the beginning of the file if I write the events on playback to the console. This may be an error from Verovio, I guess.

The peculiar thing is, that if I compare the created midi from Verovio (here I use https://tonejs.github.io/MidiConvert/ to convert it to something I'm able to read - JSON) it matches precisely the original input to Verovio (PAE).

If I then match the JSON to the event MidiPlayerJS is emmiting there are several discrepancies. I know Tonejs is showing the data a little different but it is possible to compare.

I have tried to show all differences below beginning from the list of notes. I'm only showing the events with a non 0 velocity from MidiPlayerJS.

Note 1. The duration is 0. It should be 120

Note 3. The duration is double as long as expected

Note 4. The duration is 0

Note 6. The duration is 0

I addition there is an extra note send efter End of track. This most likely comes from Verovio. It is pressent in all Verovio versions from 1.0. I guess Tonejs suppress this because it is passed after End of track

xx Peter

[grimmdude]

Hi @peterkorgaard,

Sounds like a cool project. So, MidiPlayerJS really just parrots the events that it reads in the MIDI file, so what you see is what you get for the most part. My guess is that MidiConvert isn't giving you all of the exact events found in the MIDI file for whatever reason.

However, I think I see the issue/confusion. I mentioned previously that delta is the event duration; I was mistaken. It's actually the number of ticks between it and the previous event. So, since running status doesn't send Note Off events and there's no duration information attached to Note On events, I believe what you should be using as Note Off events are the Note On events with velocity of 0, indicating that the previous note event can be stopped.

-Garrett

[grimmdude]

Hi @peterkorgaard,

FYI, MidiPlayerJS can export events in JSON format using the Player.getEvents() method. It may be helpful to compare that to what MidiConvert is generating.

-Garrett

[peterkorgaard]

This makes sense. I have checked all delta values and they are exactly the number of ticks to the previous event - as you are saying.

I have already change the playback of midi to use "note on" events with a velocity of 0 as a "note off" event. This is working perfectly and the playback is now correct. It really gives a much better performance.

Thanks for all your help, Garrett. I'm very grateful.

[grimmdude]

Great! Let me know if you have any other questions or feature requests.

-Garrett

[lpugin]

@peterkorgaard

I have already change the playback of midi to use "note on" events with a velocity of 0 as a "note off" event. This is working perfectly and the playback is now correct. It really gives a much better performance.

Was this published somewhere?

[grimmdude]

Hi @lpugin,

Thanks for your message, I've just added this note on the Readme page of this repo.

I thought about translating a Note on event with 0 velocity to Note off, but I decided not to so that these events reflect what's in the raw MIDI file as close as possible. I think for most implementations the velocity property should be considered in playback anyway.

-Garrett

[lpugin]

Thanks! I was mostly interested by where the changes made to the player by @peterkorgaard can be found. Any ideas?

[grimmdude]

Hi @lpugin,

The changes @peterkorgaard is referring to are within his own project which is using this library, not in the library itself.

-Garrett

[peterkorgaard]

Hi @lpugin There is no public access yet to our project, but there will be in the beginning of next year. We are using a combination of Verovio, soundfont-player and MidiPlayerJS together with our own software to deliver music theory. What we are doing with MidiPlayerJS is basically just to check if the volocity of the emitted midi events before playing or stopping the notes. Here is an extract of the code showing a simplified version of what the player is doing. It is a little bit ripped out of context, but the main idea is there.

`var AudioContext = window.AudioContext || window.webkitAudioContext || false;

if(AudioContext != false) { ac = new AudioContext || new webkitAudioContext;

//function that sets the instrument soundfont-player should use
setInstrument(defaultInstrument, function() {
    Player = new MidiPlayer.Player(function(event) {

        switch(event.name) {
            case "Note on":
                if(event.velocity > 0) {
                    if(!killswitch[event.track] ) {
                        noteOn = instrument.play(event.noteNumber, ac.currentTime, { "gain": (event.velocity / 127) });

                        activeKeys[event.noteNumber] = {id: noteOn.id, index: currentIndex, tick: event.tick} ;
                    }
                } else { // if velocity == 0
                    instrument.stop(ac.currentTime, [activeKeys[event.noteNumber].id]);
                }
                break;

            case "Note off":
                instrument.stop(ac.currentTime, [activeKeys[event.noteNumber].id]);

                break;

            case  "End of Track":
                killswitch[event.track] = true;
                break;
        }

    });

    playerLoaded = true;

});

} else { console.error("Browser is too old."); }`