search

JacquesLucke / animation_nodes

Node based visual scripting system designed for motion graphics in Blender.
Other
2.29k stars 342 forks source link

MIDI nodes incorrectly report MIDI file as empty #1654

Closed filibusterpie closed 3 years ago

filibusterpie commented 3 years ago

Using AN experimental build 2.2.2, accessed Jan 31. Using Blender 2.92.0 Beta.

The MIDI file is an isolated vocal part exported from 6-part piece in MuseScore3. Verified MIDI file contents via Audacity. All channels in the MIDI nodes output 0.

MIDI-1part MIDI-audacity

A "full" MIDI export of the 6pt piece works. MIDI-6part

For now I'll use the full MIDI version. Having fun using MIDI support and Blue Fox's spline tracer together.

OmarEmaraDev commented 3 years ago

@filibusterpie Can you share that file?

filibusterpie commented 3 years ago

Yup! Here's the blend and MIDI files.

AN_musicMIDI_github.zip

filibusterpie commented 3 years ago

I'm also having problems with MIDI timing on the track that does work, but I'll test + open a new thread if need be. It might stem from my music notation skill (low)..

OmarEmaraDev commented 3 years ago

@filibusterpie The files in question are of MIDI format 1. The standard specify that the first track in this format is a tempo track that only contains tempo events with no notes. That's why Animation Nodes seemingly ignored that track. However, looking at the first track in the files, apparently they also contains note events. This might also explain why the timing is off. Also notice that in the allVoiceParts file, the first track Soprano is also ignored for the same reason.

I will review the standard again and see if I can find anything that would explain this behavior.

@Clockmender @Patochun Any idea?

OmarEmaraDev commented 3 years ago

Alright. So apparently, the standard is not very clear about that. It only states that tempo events can only happen in the first track of format 1 files, it doesn't state that no other events can happen in that track. However, the established industry convention is to reserve the first track for tempo events only. Which is what Animation Nodes does and expects. MuseScore, however, doesn't adhere to this convention, hence the issue.

This was reported to MuseScore developers in the following reports. But it is yet to be solved.

https://musescore.org/en/node/207346 https://musescore.org/en/node/273557

If MuseScore doesn't fix that themselves, we can try to patch upstream ourselves. If that proved difficult, we can implement a workaround in Animation Nodes.

Thanks for the report!

filibusterpie commented 3 years ago

Ah alright then! I added a blank track on top in MuseScore - I'll leave that to designate tempo. Soprano is now accessible in AllVoiceParts, and single voice parts should work if the blank part is exported as well. I'll write a note to self re: keeping blank line for future MuseScore creations. Thanks for your help!

filibusterpie commented 3 years ago

I'll test the timing with adjusted MuseScore file.

Patochun commented 3 years ago

@OmarEmaraDev , I agree with you, the standard is not very clear about the exclusive use of track 1 for tempo only.

Clockmender commented 3 years ago

Hi Chaps,

Sorry. I am a little late getting in on this topic, various issues...

Anyway, are you aware that a MIDI file may also contain multiple tempo lines? Here is an example:

Screenshot 2021-02-03 at 14 42 24

This is Tocata & Fuge, if you were wondering, there are some 188 lines related to tempo and time signature changes in this file, so.... have you catered for that in your MIDI importer and processing, I think this is rare to have this many, but there will be many MIDI files where there are more than one Tempo and/or Time Signature. Has anyone looked at how MuseScore handles this? Bear in mind that this is allowed for in the MIDI file specification.

I have loaded this file into Reason DAW software and all the relevant info shows quite clearly in the Reason Sequencer window, so this package at least is able to interpret the MIDI file correctly. It may be that for now you need to say that multiple tempos/time signatures are not handled, but may be in the future. This obviously affects the playback time of any animation, so if you only read the first Tempo/Time Sig, any animation will go out of sequence with the sound track very quickly, i.e. at the first occurrence of a Tempo, or Time Sig change. (I know, I have been there...).

It might also be worth noting that record type 1 should absolutely NOT contain any note data at all, only file header data, such as Time Sig, Tempo, etc. Note data should always be in type 2 records (first number in the file lines). The position of any tempo/time sig change is recorded as the second piece of data in the line:

1, 142080, Time_signature, 4, 2, 24, 8
1, 158399, Tempo, 2727272
1, 158400, Tempo, 6000000

Also as far as I can tell and in all my past experience, there should not be any Tempo/Time Signature changes anywhere other than in type 1 data... Hope all this helps!

Cheers, Clock.

OmarEmaraDev commented 3 years ago

Hi @Clockmender, we do take into account multiple tempo changes and we construct a temp map from all of them. Though we completely ignore "Time Signature" events, are those important for our use case? What do they define exactly?

I have been trying to look into MuseScore's source code, but they seem to be in the process of porting code across versions or something like that, so the code is not readily explorable.

Eventually, we might do like Reason DAW and try to interrupt the data correctly.

As far as I can tell, we are fully conformant with the standard except for interpreting the tick division, where we always assume it to express the PPQN. I ignored SMPTE timecode timing because I was told it is very very very rare to find files that use that format.

Clockmender commented 3 years ago

Time signature effects the beat timing... So 4:4 is one beat per bar, 4:2 is 2 beats per bar (4 half beats per bar), not sure if that is relevant to beat timing/sound file syncing, I will do some tests, I know I had to read and account for it in my code, but just now I am not sure why I had to, it was a long time ago!

Leave it with me, I'll get back to you.

I would ignore SMPTE timecode, so old hat...

OmarEmaraDev commented 3 years ago

I submitted a patch upstream in https://github.com/musescore/MuseScore/pull/7523.

OmarEmaraDev commented 3 years ago

MuseScore 4.0 no longer have a MIDI exporter, at least for now. And no new 3.x releases will be done, so our patch will not be merged. There isn't much we can do, they have the patch and the rest is up to them. Closing this for now.