FluidSynth / fluidsynth

Software synthesizer based on the SoundFont 2 specifications
https://www.fluidsynth.org
GNU Lesser General Public License v2.1
1.85k stars 256 forks source link

Support for FLAC compressed samples? #605

Closed mxmilkiib closed 3 months ago

mxmilkiib commented 4 years ago

SF3 uses the Vorbis lossy format, which, whilst offering a reduced file size, comes with a reduced sample quality that would often be unsuitable for a professional audio environment.

SF4 is essentially a similar modern variation on SF2, albeit using losslessly compressed FLAC audio samples, offering a balanced middle-ground between SF2 and SF3.

(The SFZ format already supports FLAC samples.)

There is scant support for SF4 as of yet - there is a conversion script available, https://github.com/cognitone/sf2convert - but does seem an eminently sensible idea.

BeRo1985 commented 1 year ago

Even if "proper" SF3+FLAC became a thing, 99% of synths will still decompress all the samples when loaded, using the same amount of ram either way.

So I also have quite a few SF2 files which are larger than 500MB, or even some with several GBs, of which I also almost always use only one to a few instruments from the tens of hundreds or thousands of instrument patches in these SF2 files, where it would be very counterproductive to have to load the whole soundfont into RAM and decompress it just to use a few instrument patches from it. And besides, I can only agree with what sagamusix has already said about the file format recognition headache then.

In other words, your idea is well-intentioned, but poorly thought out and not practical in the real world, which is why it is not feasible.

chipweinberger commented 1 year ago

Absolutely not, because then you first have to decode the ENTIRE thing to memory, instead of just the instrument(s) you actually need

Who says you need to load it all in memory?

After you get your original SF2, you can just load the instruments you need.

Programs don't even need to keep the entire SF2 file in storage, if they want to make that optimization. Just discard bytes for instruments you don't need as you decode. Flac decode is quick and requires little ram.

*.sf2flac files cover all your needs, and are a very simple standard.

How should it now decide how it should be treated?

The *.sf2flac file extension, of course! The main way we know how to treat most files.

A more complicated flac standard would really need to justify its existence in light of sf2flac already existing. (yes it does exist now, as of my first comment :)

chipweinberger commented 1 year ago

To give some perf numbers,

Ocarina Vib.sf2 : 133 MB
Ocarina Vib.zip : 119 MB
Ocarina Vib.sf2flac : 49 MB
flac decode time: 1.25 s (M1 Macbook Air)
flac encode time: 1.45 s (M1 Macbook Air)
mirabilos commented 1 year ago

Chip Weinberger dixit:

flac decompression time: 1.35 s (M1 Macbook Air)

Now this is unfair.

Do it on a RPi 2 with 1 GiB RAM in total.

Without swapping, because that kills the card.

This is the scenario I was looking for (and which I was actually supporting an actual user on, with MuseScore).

bye, //mirabilos --

den AGP stecker anfeilen, damit er in den slot aufm 440BX board passt… oder netzteile, an die man auch den monitor angeschlossen hat und die dann für ein elektrisch aufgeladenes gehäuse gesorgt haben […] für lacher gut auf jeder LAN party │ damals, als der pizzateig noch auf dem monior "gegangen" ist
sagamusix commented 1 year ago

The *.sf2flac file extension, of course! The main way we know how to treat most files.

A lot of well-written applications do not need to rely on file extensions. Not every data stream that you may want to feed into an application is going to have a filename associated with it. Your "simple" design is making software that doesn't care about filenames needlessly complex or impossible to write.

If you really want an SF2-derived format which stores all samples in a single continuous FLAC stream, it already exists: It's called SF2PACK. No need to re-invent it.

BeRo1985 commented 1 year ago

And besides, still the completely wrong approach, file format design-wise. This only makes loading more complicated, if you also have to put a FLAC decoder in front of it so that it is also able to skip byte ranges, and what if a SF2 loader implementation also wants to do random-seeking, e.g. to reload on-demand samples?

Side info: I have implemented my own FLAC decoder from Scratch for Sobanth (and for my own WIP DAW). But not every SF2 compatible engine developer is able to implement an own FLAC decoder or would be able to extend or hook an at least existing FLAC decoder so that it can skip byte ranges accordingly. And besides FLAC decoding then still costs CPU time unnecessarily, also for the byte ranges to be skipped.

The bottom line is that this idea is simply not practical, even if it was well intended.