[feat] Support for MIDI (.mid) files on Teia

[ryangtanaka-org]

Describe the problem

MIDI is an underappreciated medium - it's lightweight, code compatible, and opens up a lot of opportunities for experimentation - perfect for TEIA to support its mime type, imo.

This was a long time ago but I did some work with this person for my grad school composition project - MIDI is interesting because it lets you visualize and manipulate the outcomes in ways that you can't with regular sound files. Might give some ideas about visualization as well. (He was doing this since the 80s, FYI.)

http://www.musanim.com/Player/

Describe the solution you'd like

https://github.com/cifkao/html-midi-player

MIDI is a very archaic tech so a lot of it is unsupported but this one seems to be active -- might be fun to play around with the visualization aspects too 😎

Alternatives considered

No response

Additional context

No response

[melMass]

Hey Ryan

It's an interesting topic. The thing is that this is already doable using x-directory but like GLB it could indeed have basic support. What would a "generic" midi player look like?

I personally like the idea that the artifact is the output file directly (MIDI, GLB) and not a web project export

[ryangtanaka-org]

Hey, thanks for taking a look! Yeah, the html thing was just a suggestion (might be something we use for our own project) but if Teia wants to display the .mid files in a more neutral way I think that's totally cool too.

Most people probably don't know about the file format but for musicians (especially composers) it's an important tool for music-making that often doesn't get the attention it deserves - it'd be great if we could create a space for people to mint their works for posterity - I know a lot of folks in the classical music realm would probably appreciate it, if anything.

[melMass]

Oh no, on our side it will be handled using some library like the one you provided. I was making the analogy to glb as this would work like them: stored in the native format on IPFS (we use the google viewer I linked you to on discord for that).

But I'm still unsure of the expectations of a basic MIDI player like is sound mandatory? Does it have to have a play function then or just display a table of notes?

[ryangtanaka-org]

Sorry took a while to get back - MIDI is sort of a weird hybrid between computer code and sound so there was never really a "standard" way to "display" them, if you know what I mean. The interesting thing is that because it's just a series of commands, the way the music is rendered changes from machine to machine, depending on what type of soundcard or soundfont library they're using client-side. (Kind of fits in the ethos of Web3's decentralization mantra, but that's another issue altogether.)

But as far as consumption goes, it's fairly important to have some sort of sound playback - audio players used to support them but unfortunately I don't think html5 supports it right now. (Would've been the easiest way to do it.)

Visualizations (piano rolls, notation, graphical visualizers) are optional, but that's more of a curatorial decision because I think some musicians would like it, but some purists out there might say that it should be for sound only.

Either way - at this point I don't think those details really matter since this idea is so niche I kind of doubt anyone has even any real opinions around it right now. Just having anything would be amazing because there's probably millions of files out there sitting around on people's computers sitting around doing nothing.

One "practical" application of MIDI as a consumer product is karaoke - I used to go to parties where they would play these files and sing along to their favorite tunes in a fun, customizable way. Or just cool to have all the "data" of the music that you happen to like so you can transform it or learn from it. Still lots of potential for experimentation, imo - and if there's anywhere that might do something interesting with it, it'll probably be here.

Some sites convert MIDI to MP3s to make it more palatable to Web2 standards but I don't think that's what we should do here, just in case.

https://bitmidi.com/

[ryangtanaka-org]

Oops, it's been a while - I didn't realize that a lot of browsers depreciated MIDI from the audio tag so it wasn't as easy as originally thought. Kind of disappointing, to be honest.

After asking around and thinking it over, I think it's best we not mess with the "visualization" parts of it because the visuals are something that the artist probably wants control over - so it's better to just leave it as pure as we can. I think something like this with a minimal UI, minus the tempo slider since that should be set by the musician, not the user.

https://grimmdude.com/MidiPlayerJS/

Because getting the client-side native sounds may be difficult now, we should probably just use a standard soundfont as well - the one from the example above will do. (It's going to sound a bit tacky in many cases, but in the hands of someone who knows how to manipulate the sounds properly it can be made to sound decent - and if the user wants something customized, they can download their own soundfonts as well.)

https://github.com/danigb/soundfont-player

And the last thing we need is a standard, which should be General MIDI Level 2, which was the last time the standards around this stuff was updated. The name of the standard (including the name of the soundfont used) should be somewhere inside of the minting process so that musicians know what they're working with. We want to keep this consistent so that it keeps everyone's expectations on the same page.

https://en.wikipedia.org/wiki/General_MIDI_Level_2

So what this allows for is for musicians to upload their MIDI files onto IPFS, then that link can be used for others to build other things upon them, visualizations included. It'll be pretty experimental for a while but I think it'll be a draw for a lot of musicians out there who have a lot these files sitting around on their HD anyway. If it's not too much work think it's worth at least a shot, anyway.

[ryangtanaka-org]

Here are some samples. Note that the instruments for GM typically are 0-127 with standardized instrumentation.

https://en.wikipedia.org/wiki/General_MIDI

midi.zip