search

ethman / slakh-generation

A project to synthesize massive amounts of multitrack audio data from MIDI.
http://www.slakh.com/
MIT License
55 stars 4 forks source link

Ambiguous mapping between MIDI to drum instruments #6

Open MZehren opened 1 year ago

MZehren commented 1 year ago

Hello,

After listening to some tracks from Slakh, I have the feeling that there is a mismatch between general MIDI pitches and the mapping Kontakt uses for the drum instruments. For example, pitch 53 refers to "Ride Bell" in general midi and "Hi Tom Right" in Kontakt (at least in my version: Kontakt Factory 1.4.2, patch Band/Pop Kit).

This might mean that the synthesized audio is not representative of the intention of the annotator of the midi file if this person was using general midi. Moreover, pitches 60 to 71 seem to make Kontakt play sequences of notes instead of a single one, which might lead to synthesized audio even more different (and less realistic?) than the intended one.

Is this intended? The differences between general midi and Kontakt I identified are pitches 52, 53, 54, 58, 60 to 71.

ethman commented 1 year ago

Yes, there 100% is a mismatch between Kontakt and general MIDI rules for percussion. If memory serves right, I think none of the Kontakt instruments honor the drum events as defined in MIDI. I tried mapping a few that I found manually here, and after I made this project a colleague pointed me to the Kontakt documentation that outlines their drum mapping (I don't know where to find it at the moment, but it's on the web somewhere). So, yeah, it's a known issue with the Slakh dataset unfortunately.

I am not working on this project anymore, but if you want to propose some new rules for re-mapping MIDI to Kontakt drum notes, I will happily merge the PR. Also, I believe a different colleague mentioned that migrating this codebase from python2.7 to python3 was minimally painful, if you feel so motivated to generate more data.

MZehren commented 1 year ago

Thank you for your answer!

After more investigation, I found that there is more than Kontakt using their own mapping of MIDI pitches to drum instruments instead of General Midi. Actually, it is each synthesizer within Kontakt that use a different mapping as well. (i.e., Pitch 35 is mapped to "Tones", "Clap", and "Bass drum" from "Street knowledge kit", "Funk kit", and "Pop kit respectively). Considering that this dataset is used to train transcription algorithms (e.g., Ishizuka et al. 2021, Simon et al. 2022,) I think it would be very valuable to provide a disambiguated MIDI file corresponding to the audio.

(I don't know where to find it at the moment, but it's on the web somewhere)

After quite a long search I could not find this. Is there a chance you could find it easily? Otherwise I will try to reach Native Instruments. I will also send an email to the authors of the works cited earlier, they might have a solution.

I would be happy to provide a complete mapping if I can create one. In the meantime, I attach a reverse engineering mapping I conducted on the 3 synthesizers I have with Kontakt factory: MIDI_to_kontakt.txt

Cheers!