spessasus commented 1 month ago

Since SFe isn't really backwards compatible (see RIFF64, TSC or changing words to dwords, I propose to revise the file structure to remove all limits. It is heavily inspired by the DLS file structure.

My proposal

file extension

.sft - short for "SoundFonT"

Definitions

XXXX:YYYY is XXXX is the chunk type, and YYYY is the fourcc at the start of data (like "sfbk" in sf2)
LIST chunks contain multiple RIFF chunks inside.
Chunks with 64 instead of the last two characters are 64-bit.
FLOAT is a signed 32-bit IEE-754 floating point number.
ALL NUMBERS ARE LITTLE ENDIAN

The INFO chunk

There's a defined INFO chunk that can be added to almost everything. It is described below.

All chunks here are 32 bits (no need for more than 4GB of text, come on)

All text chunks must use the utf-8 encoding!

All chunks within the INFO chunk are strings unless explicitly stated otherwise.

LIST:INFO
- INAM - Required. Name of the object that this list is in.
- ICOP - Optional. Copyright of the object.
- ICMT - Optional. Comment/description of the object.
- IENG - Optional. Creator of this object.
- ICRD - Optional. Date formatted as ISO-8601 string to ensure that software can read it as a date regardless of locale.
- ISFT - Optional. The software used to create or edit this object.
- LIST:PROP multiple - Optional. A custom property defining something (like "MIDI system": "Yamaha XG"). There can be multiple of these chunks in the info list.
- PNAM - Property name as a string.
- PVAL - Property value as a string.

This chunk is denoted as INFOLIST in the format below, optionally listing additional chunks for this specific field.

Curve

SFZ defines the concept of curves. Let's add them here!

The Curve is defined as an X number of points. The First one has to be 0, the last one has to be 1. The points are evenly spaced apart and interpolated LINEARLY.

Curve chunk

Here's a curve chunk

LI64:CURV - a 64-bit RIFF chunk
- PT64 - point data, a 64-bit RIFF chunk
- pointsAmount - an unsigned 32-bit integer describing the number of points in the curve.
- [points] - a sequence of FLOAT numbers one after another, describing the points. [points] must take exactly pointsAmount * 4 bytes.
- INFOLIST - the optional INFO list to describe this curve (for example, by naming it GS Volume Curve)

for example, for a basic linear curve, you would do

points amount = 2
point #1 = 0.0
point #2 = 1.0

Modulator Index list

A simple list describing the used modulators. Used in presets, instruments, and the default mod list.

MODS
- modsAmount - an unsigned 32-bit integer describing the number of modulators in the list. The values below are NOT RIFF chunks, just numbers one after another
- [indexes] - a sequence of 32-bit unsigned integers one after another, pointing to an index within the MODL list in the main INFO chunk. [indexes] must take exactly modsAmount * 4 bytes.

Sample

Here's a chunk that describes a single sample.

LIST:SAMP - a single sample.
- INFOLIST - the INFO list describing the sample.
- MKEY - the original MIDI key number of the sample. a 16-bit unsigned number.
- GAIN - the gain adjustment to the sample. A FLOAT number.
- TUNE - the tuning for the sample, expressed in cents. a 32-bit signed integer.
- SDIX - sample index. This chunk is 16 bytes long. The values below are NOT RIFF chunks, just numbers one after another
  - indexStart - The first eight bytes form the 64-bit unsigned start index in bytes starting from the first byte of the sample data chunk.
  - indexEnd - The last eight bytes form the 64-bit unsigned number of the sample end index (the last byte of the byte-stream) starting from the first byte of the sample data chunk.
- TYPE - a 16-bit unsigned integer describing the audio type. See audio types
- LIST:LOOP - the loops for this sample. OVERLAPPING LOOPS ARE NOT ALLOWED!
  - LOOP multiple - a single loop. The values below are NOT RIFF chunks, just numbers one after another
  - loopCount - a signed 16-bit integer describing the loop count of this loop. 0 means infinite loops and -1 means "exit loop on key release"
  - loopStart - an unsigned 64-bit integer describing the loop start index IN SAMPLES relative to first sample inclusive.
  - loopEnd - an unsigned 64-bit integer describing the loop end index IN SAMPLES relative to first sample, exclusive.

Audio types

The TYPE chunk within a sample chunk describes the type of bitstream the sample uses.

Currently, I can think of these types:

0 - WAV - a complete WAVE file. Only PCM encoding would be allowed. Any bit rate and any sample rate.
1 - OGG VORBIS - a complete OGG vorbis file, like the Werner SF3 version.
2 - FLAC - a complete FLAC file.

More can be added later (up to 32k!)

Up to two channels of audio are allowed. They shall be interpreted as effectively two samples with exactly the same generators, except for pan which is forced to -1 on the left sample and 1 on the right.

Generator list

Here's a generator list.

GENL - generator list. The values below are NOT RIFF chunks, just numbers one after another
- generatorsAmount - a 32-bit unsigned number of generators in this chunk.
- [generators] - a bit stream of generators. A generator is defined as a 6-byte stream.
  - type - a signed 16-bit generator type. The generator types are to be defined
  - value - a FLOAT number with the generator value. Four bytes.

Another thing, since the value is a float now, I propose changing timecents to seconds (or miliseconds?). It would make things A LOT easier. Also making pan be -1 to 1.

Modulator

The modulator is something I'm unsure about, but it has to implement the following:

modulate the sample chosen (for example, MIDI cc#1 would change the sample played)
more transforms
proper linked modulator description
make use of the custom curves as defined in the main INFO chunk
it also has its own INFO chunk (since the modulator list is global and reusable)

Instrument

Here's a chunk that describes a single instrument

LIST:INST - the instrument
- INFOLIST - the info list describing this instrument.
- FLAG - a two-byte long chunk, forming a 16-bit unsigned flag number. Currently undefined.
- LIST:ZONE - the zone LIST.
- KEYR - the MIDI key range as a 16-bit number. the first byte is the minimum MIDI key, and the second byte is the maximum. BOTH INCLUSIVE.
- GENL - the generator list
- MODS - the modulator index list
- INFOLIST - optional - the info about this zone.
- SIDX - optional - sample index within the sample list. a 32-bit unsigned number. The lack of this chunk makes this a global zone.

Preset

Here's a chunk that describes a single preset

LIST:PRES - the preset
- INFOLIST - the info list describing this preset.
- BANK - a four-byte long chunk. The First two bytes form a 16-bit unsigned bank MSB number and the second two form a 16-bit unsigned bank LSB number.
- FLAG - a two-byte long chunk, forming a 16-bit unsigned flag number. Zero means nothing, 1 means a "drum preset." Other flags can be added later.
- LIST:ZONE - the zone LIST.
- KEYR - the MIDI key range as a 16-bit number. the first byte is the minimum MIDI key, and the second byte is the maximum. BOTH INCLUSIVE.
- GENL - the generator list
- MODS - the modulator index list
- INFOLIST - optional - the info about this zone.
- IIDX - optional - instrument index within the instrument list. a 32-bit unsigned number. The lack of this chunk makes this a global zone.

File format

RF64:snft - main RIFF chunk (64 bits)
- INFOLIST - The INFO LIST. This one has the additional chunks below:
- LIST:MODL - The global modulator list for the SoundFont. All modulators are described here
  - SMOD multiple - A single SoundFont Modulator. The structure is to be decided, but I'm thinking of the DLS articulator structure. Also with an optional INFO list.
- LI64:CRVS - the defined curves list.
  - LI64:CURV - a single curve. Defined in Curve Chunk
- DMOD - default modulators, applied to all instruments.
  - MODS - the Modulator index list.
- LI64:SFDT - the soundfont data (64 bits)
- LI64:PRSL - preset list (64 bits)
  - LIST:PRES multiple - a single preset
- LI64:ISTL - instrument list (64 bits)
  - LIST:INST multiple - a single instrument
- LI64:SMPL - the sample list (64 bits)
  - LIST:SAMP multiple - a single sample.
- SD64 - the SAMPLE DATA

Sample data

THe SD64 contains all the sample data. All the data is a bunch of bitstreams one after another.

For example, if the first sample was WAVE and a second one was FLAC, the bitstream would start with the WAVE file and just after that there would be the FLAC file pasted in.

Conclusion

What do you think of this format? Is it any good?? Let me know!

stgiga commented 1 month ago

You have some great ideas.

spessasus commented 1 month ago

Thanks, @stgiga ! There are a few things I thought of but missed when writing above though:

modulators

Source can be one of 4 types:
- MIDI CC num (like in SF2)
- special controller: like key num, velocity, channel pressure, poly pressure
- RPN value: like pitch bend range, coarse and fine tuning, modulation depth. These 4 would also have the stock modulators built in
- NRPN value: a 14-bit value of both LSB and MSB: it can allow us to program in stuff like GS vibrato!!!

Maybe even somehow program in sysEx messages? (like if manufacturer ID is this and nth byte is this and that, then take byte X as the value). It is probably a bit too complex though (or is it? ;-)

more transforms:
- source has its own transform
- control (secondary source) has its own transform
- output has its own transform
- absolute value
- absolute value * -1
- floor
- ceiling
- modulo by something
- clip output to given range (for example a bipolar linear curve which only allows values from 0 up, effectively disabling the modulator if source is below half)
changing type
- add (default)
- subtract
- multiply
- divide
- override (only one modulator can be used on this generator if it overrides to prevent race conditions)

samples

Sample interpolation types. I can think of 4 now:
- linear
- nearest neighbor: this would make things like square wave only require one sample without it losing its "sharpness" when played at low tones
- 4th order
- 7th order (would be default)
Loop types, it would be a flag that goes after loopCount:
- forwards (normal)
- back and forth (one direction change is considered one loop)
- backwards

Generators

initial attenuation would allow negative values (why not?)
key to decay and hold removed (replace with modulators)
decay changed to express the actual decay phase time, not time to -100dB

custom chorus

can be multiple, with info, a CHRL - chorus list chunk
you can set the left and right channel delay line count and their values (and lfo modulation depth and rate)
selected via a generator (chorusType and it would be the index in the CHRL chunk)

stgiga commented 1 month ago

Funnily enough, having those extra fields can improve Roland SC-88Pro and SC-8850/8820/SC-D70 Insertion EFX (CC94), as well as open up whatever fancy stuff MU2000EX XG3 does. So I'm OK with implementing this, and I've gotten the other team member involved.

spessasus commented 1 month ago

Funnily enough, having those extra fields can improve Roland SC-88Pro and SC-8850/8820/SC-D70 Insertion EFX (CC94), as well as open up whatever fancy stuff MU2000EX XG3 does. So I'm OK with implementing this, and I've gotten the other team member involved.

Well, that's what I was thinking of, specifically the 88Pro. I actually started my MIDI and soundfont adventure by discovering MIDIs made for 88Pro. So recreating these perfectly would be amazing!

stgiga commented 1 month ago

My first major SoundFont was an 88Pro SoundFont, and these days Tyroland (and some others, but on back burner because college) has good SC-8850 support created from the Tyros4+JV1010 samples in a long process.

The project lead got into SC-88x support because of my efforts (which have roots as far back as 2015), because I hated seeing broken songs. So I think even the project lead would be interested.

sylvia-leaf commented 1 month ago

@spessasus Thank you for the draft for the SFe rework. Sorry for taking so long to write my response.

Naturally, because you came up with a complete re-structuring of the SFe specification, I've got a lot of feedback to give you.

You are probably right when you say that SFe isn't 100% backwards compatible with SF2.04. We haven't managed to confirm that everything implemented is done in a compatible manner.
This is why we're working on a few different versions of the SFe format:
- SFe32: completely backwards compatible with SF2.04
- SFe32 with TSC: not technically different to SFe32, includes TSC and other esoterical stuff that isn't compatible with SF2.04
- SFe64L: sticks with the size limits and is structurally very similar to SF2.04, we might move incompatible SFe32 stuff to SFe64L
- SFe64: the full version of SFe that has extended size limits and will eventually become unlimited with future versions
Talking about the DLSification of the SF format: we don't want to reinvent the wheel, I think that it is possible to get things working in a backwards compatible manner with SF2.04.
- We've actually been working on different evolutions of the SF format. SFe is just one of them; there was another one called SFi which worked by not modifying the format, and intelligently activating features via sample naming or similar.
- I think that we should adopt your draft suggestion as a different evolution of the SF format called SFt. We won't abandon SFe, as SFt is a more difficult system for programs to implement.
So, why didn't we use the file extension .sft? The answer is that "soundfont" is trademarked by Creative and should not be used to describe improved formats such as SFe, or non-SF based formats such as SFt.
With the curves idea, you mention that you took it from SFZ. Full SFZ opcode translation support is actually part of our plans for a future version. Currently our roadmap (including previous versions from others) looks like:
- Version 1.xx: First version of the SF format for AWE32 use.
- Version 2.xx: Public release of specification by Creative.
- Version 3.xx: Implement compression in a widely compatible manner. Done by Werner.
- Version 4.xx: Merge as many SF2.04 extensions as possible into one.
- Version 5.xx: SFZ opcode development, release full version of SFe64.
- Version 6.xx: Further size limit removals for SFe64.
It's not clear whether SFt is a RIFF-based format, or a completely new system. If it's non-RIFF, then I tried that. We called it "NGMSS" (next-gen music sample system), it didn't go anywhere.
If SFt is RIFF-based, then it may be used as the basis of SFe64 starting from version 5. SFe should also refrain from being too different from SF2.04.

You do have some good ideas, and I'll be working on version 4.00.6 of the specification, which splits the versions of SFe apart. Please check the branches of the repository for a preview. If you want to be part of the SFe team, then please tell me!

sylvia-leaf commented 1 month ago

Forgot to add this. The many features that you mention that are incompatible with SF2.04 such as TSC, DWORDs and RIFF64, are mostly absent in the SFe32 specification.

Please take a look at the SFe32-only branch of the repository for the compatible version of the SFe specification.

Also, if you have any suggestions about where we can place the TSC implementation info so we can make it clear that using TSC will make your file incompatible with some players? Or should we scrap TSC all together?

We'll probably remove TSC implementation info temporarily for 4.00.6. We'll also plan to bring it back at a future date, maybe as part of SFe64L.

stgiga commented 1 month ago

I DO NOT want TSC scrapped.

spessasus commented 1 month ago

What's the point of TSC?

Just use 64-bit riff chunks like I suggested and then you have practically no size constraints.

Since TSC is already fundamentally incompatible with sf2, why not just skip to 64?

stgiga commented 1 month ago

What's the point of TSC?

Just use 64-bit riff chunks like I suggested and then you have practically no size constraints.

Since TSC is already fundamentally incompatible with sf2, why not just skip to 64?

I mean, it was to make 32bit at least not be a complete rewrite for more samples.

sylvia-leaf commented 1 month ago

Alright, there seems to be some disagreement about the TSC mode. Let me try to understand what's going on:

Stgiga has identified the advantages of the kinds of features found in SFe32 and SFe64; they said that TSC mode in particular is useful to increase the size of 32-bit banks.
However, Spessasus questions the compatibility of TSC mode with legacy SF2.04 players. Because TSC mode hasn't been formally tested for SFe, they may be right or not.

Right now, I'm going to propose this compromise:

TSC mode will not be removed completely from the SFe specification.
However, TSC mode will be limited to SFe64L, which is the extension of SFe32 designed to provide many of the advantages of SFe64 without rewriting the code of SF players significantly.
Confusingly, SFe64L will encompass both 32- and 64-bit banks using RIFF and RIFF64 headers, as well as the corresponding chunks.
If you have any suggestions about the final name of SFe64L, then please share them. "SFe64L" was always intended as a temporary name.
SFe32 will only have things which can definitively be confirmed to be compatible with SF2.04; anything that fits in the SFe32 infrastructure that is incompatible goes in SFe64L.
We have access to a computer that uses a Sound Blaster X-Fi Titanium. If SFe test banks cannot be loaded onto the sound card, it is suitable for SFe64L and not SFe32.
The next step is to test TSC mode on physical Sound Blaster hardware; if it doesn't work on physical Sound Blaster hardware, then it will go to SFe64L.

Stuff to do:

Find documentation about TSC mode from saga/OpenMPT, and link to it in a future version of the SFe32 and/or SFe64L specification.
Test 32-bit banks that implement TSC mode on the Sound Blaster X-Fi Titanium.
Decide where to mention TSC mode in the specification.
Test other proposed SFe features on the Sound Blaster X-Fi Titanium.

spessasus commented 1 month ago

Alright, let's sum my thoughts up:

TSC is putting SDTA at the end of the file. This is fundamentally incompatible with SF2 and therefore with all Sound Blasters and softsynths. So, a rewrite. Not to mention that there's no point in TSC with SB as they can load up to 30MB (or something like that) soundfonts. So the only advantage it brings (larger SDTA limit) is nullified.

RIFF64 is making some chunks use 64-bit length rather than 32-bit and changing the 2 last letters of fourCC to 64 (in ASCII). This is also fundamentally incompatible, so a rewrite.

Now, here's what a "rewrite" means for SpessaSynth for both of these:

TSC: Reorder the file parser and completely change the sample loading logic as when reading SHDR, the player already expects the sound data to be available. This is a major rewrite.

RIFF64: change the RIFF parser to check if the last 2 characters are 64 and then reading length as 8 bytes instead of 4. This is a very minor change that I can implement in 2 lines of code. In fact, here is the new code:

export function readRIFFChunk(dataArray, readData = true, forceShift = false)
{
let header = readBytesAsString(dataArray, 4);
let sizeByteLength = header.substring(2) === "64" ? 8 : 4; // this line is new
let size = readLittleEndian(dataArray, sizeByteLength);     // this line is changed
let chunkData = undefined;
if (readData)
{
    chunkData = new IndexedByteArray(dataArray.buffer.slice(dataArray.currentIndex, dataArray.currentIndex + size));
}
if (readData || forceShift)
{
    dataArray.currentIndex += size;
}
if (size % 2 !== 0)
{
    if (dataArray[dataArray.currentIndex] === 0)
    {
        dataArray.currentIndex++;
    }
}
return new RiffChunk(header, size, chunkData);
}

As you can see, TSC is very much a major rewrite, unlike the 64-bit extension (at least in SpessaSynth's case). And not to mention, come on. All modern computers are 64-bit. The only major exception would be the hardware soundblasters which don't take advantage of TSC anyways.

And even then, the 64-bit mode could work for 32 bit machines. The spec could add the following:

For 32-bit software, 64-bit chunks have to be checked. Any of the top four bytes is not set to 0, this means that the chunk size is above 32-bit maximum and the file should be rejected as incompatible with the architecture. If the top four bytes are all set to zero, the chunk must be read and played normally.

sylvia-leaf commented 1 month ago

So, you've shown that it is a lot easier for a software developer to adapt to 64-bit than to use TSC mode.

Therefore, we're going to be moving TSC mode to its own specification separate from SFe. It will require the player to support SFe, but SFe won't require it. Version 1.0 of this specification should be released alongside of SFe 4.00 or 4.01.

To facilitate this method of developing features, we are working on giving the ISFe chunk its first use, as a "feature flag" system. The specific flag used for TSC will be well-defined, but the information on TSC will be absent from SFe specifications until we've sorted out where TSC mode should go.

sylvia-leaf commented 3 weeks ago

So, you've shown that it is a lot easier for a software developer to adapt to 64-bit than to use TSC mode.

Therefore, we're going to be moving TSC mode to its own specification separate from SFe. It will require the player to support SFe, but SFe won't require it. Version 1.0 of this specification should be released alongside of SFe 4.00 or 4.01.

To facilitate this method of developing features, we are working on giving the ISFe chunk its first use, as a "feature flag" system. The specific flag used for TSC will be well-defined, but the information on TSC will be absent from SFe specifications until we've sorted out where TSC mode should go.

sylvia-leaf commented 3 weeks ago

For now, TSC mode has been moved to a separate specification independent of SFe. However, this may change in the future. Sorry for posting the same reply twice!

SFe-Team-was-taken / SFe

SFt Proposal #1: Initial Idea #6

My proposal

TOC

file extension

Definitions

The INFO chunk

Curve

Curve chunk

Modulator Index list

Sample

Audio types

Generator list

Modulator

Instrument

Preset

File format

Sample data

Conclusion

modulators

samples

Generators

custom chorus