How to deal with old Scratch 1.4 sounds?

[towerofnix]

There's an old Scratch 1.4 block – I'll call it "MIDI instrument" since that's its op code (midiInstrument:). It's Scratch 1.4's version of the "set instrument" block.

In Scratch 2.0, the block looks like this:

The only visible difference is that one block has the obsolete colour and the other has the Sounds category colour, but they are stored differently in the project files and they do behave differently.

Scratch 2.0 has very few instruments in comparison to 1.4 – only 21 vs. 128! (No factorial.) So, to account for the limited sound options, Scratch 2.0 uses a mapping of 1.4 instrument IDs to 2.0 instrument IDs. (That's defined here.)

However, I think it's not very debatable to say that this isn't the right thing to do. Any old 1.4 projects that used instrument blocks are basically broken by this, and any future (well, current) musical projects are relatively more limited now.

The simple way to handle this would be to use the very same map along with a new sound primitive definition in the VM, but that's only a semi-fix. On the one hand, that'd make those 1.4 projects behave the same way they have for the last three-ish years. On the other hand, the projects still "don't work" – they still don't behave in the same way as their old user-created versions.

Arguably, this isn't even a problem – Scratch 2.0 has already been the current version for three and a half years (even longer when 3.0 is released!). Chances are very few people, if any, still care about the old versions of the projects, or the old instrument options. (I was hardly even around then – I certainly didn't care about those projects!)

But the real best solution is likely to come from the official reason these sounds aren't available anymore. Here's some text from an old post by Lightnin:

Unfortunately, for various reasons, Scratch 2.0 can't use those [MIDI] synthesizers, so we have to build our own synthesizer and musical instrument library.

Here's the limitations he's talking about:

We can only use SoundFonts that are public domain or available under a license that allows us to distribute them freely, such as a Creative Commons license.

We're limited in size. The entire library should be no more than two megabytes. Assuming we include 25 instruments, each instrument should be about 80k in size.

So one problem is file size. At this point I think it's alright to sacrifice a bit more data – remember, connections are always improving and browsers do cache files well!

So the real issue here is licensing, and a lack of the sounds.

I'd be interested in seeing how collecting those sounds might go. At this point, the Scratch community is larger than ever, so gathering the sounds might be easier than it was back in 2013 (there's now 10 times more active users!). I guess one of the problems might be consistency of quality, though – obviously sounds from different people are likely to have subtle recording/accuracy differences. The real best bet would be to have one person, or an organized group of people, record every sample, but 128 instruments (of which quite a few may be more difficult to get! There's quite a lot of synthesizers and such..) is quite a daunting number..

I wonder if there's some MIDI libraries already created online that have licenses that would be compatible with Scratch? That'd probably be the best way to get 128 accurate MIDI instrument samples.. :)

TL;DR revisiting the old MIDI sounds might be nice.

But I'm not the professional audio person here, nor a lawyer, so.. I'm really just bringing this up for discussion!

(edit: I'm assuming scratch-audio is the best place to put this!) (edit2: This also applies to old drum sounds, I think.)

[ed-cooper]

Is it not possible for Scratch 3 to use the default MIDI synthesizers on the computer then?

(Btw happy 2017! 🎉)

[towerofnix]

I suppose, on the VM/blocks side of the project, extending the normal "set instrument" block to support the 128 MIDI instruments (coming to a total of 149 instruments) could work.

[ericrosenbaum]

Thanks for your comments on this! The way we will handle the instruments in Scratch 3.0 is at an early stage, but we are likely to maintain compatibility with the smaller Scratch 2.0 set of 21 instruments.

The original larger set of 128 instruments imitated the existing 'general MIDI' standard. The updated set of only 21 instruments reflects the design philosophy of Scratch: a balance between simplicity (a small number of options that is friendly and not overwhelming) and flexibility (a diverse set of options that you can use lots of different ways).

The actual implementation of the instruments will be done in the browser using its webaudio functionality, so that the experience will be consistent across platforms.

Right now what we have is a sort of temporary placeholder functionality, which plays notes using an existing soundfont library. These soundfonts are high quality, but far too large in file size for our purposes, and for the moment they load on demand (i.e. at the moment the 'play note' block or 'set instrument' block runs, if the soundfont has not yet loaded, it loads, causing a delay of a few seconds). Other (temporary) limitations include the fact that we cannot route these sounds through all of the audio effects, we cannot set the volume, and we cannot sustain the sound longer than the sample duration. All this will change once we decide on a new approach that gives us high quality sounds with a reasonable file size.