Slow HDD performance can block FluidSynth playback / How to load entire SF2 to memory on open()

[kcgen]

FluidSynth 2.1.5, Library Version

Extremely slow disk IO performance can act like a denial of service on FluidSynth after the initial load of the SF2 file, where the first playback of a demanding multi-voice sequence will cause FluidSynth to drop those sections of audio and simply produce no output while the loads are blocking due to slow disks seeks and reads.

Note that "spinning rust" HDD seek times are typically > 10ms, and can grow into 100's of ms on heavily loaded systems.

Expected behavior

Because FluidSynth is a realtime audio player/synthesizer, it must keep up with real time rates and should protect itself from and common influences that can harm its performance, such as slow HDDs.

Ideally a flag can be provided to tell FluidSynth to "read the entire SF2 into memory"; and this should probably be the default unless developers (or users) are confident that their IO subsystem can keep up with realtime requests.

(Yes, aware of http://www.fluidsynth.org/api/fluidsynth_sfload_mem_8c-example.html, is an option, but that feels like quite a hack when FS could internally just allocate a large enough buffer to hold the SF2 on load).

[mawe42]

FluidSynth loads SF2 Files into memory by default, unless you have explicitly told it not to via the dynamic-sample-loading setting. So I would be very surprised if the problems to you are experiencing are fluid io related.

[derselbst]

Just like Marcus already said, SF2 are fully loaded into memory by default. The file-handle is even closed after having done so:

https://github.com/FluidSynth/fluidsynth/blob/97615ef2cff5b5382eb128e5d04ab680430835a5/src/sfloader/fluid_defsfont.c#L522

There will only be HDD activity when synth.dynamic-sample-loading is enabled.

To me, it doesn't make sense why loading from a ramdisk fixes all your problems. Please provide a profiler report (or similar) to prove that this issue is located in fluidsynth. Otherwise I have to consider this report to be invalid.

[kcgen]

We've recently integrated libfluidsynth in dosbox-staging; so to be clear we're not using the standalone executable (if loading behavior differs).

Here's the report:

https://github.com/dosbox-staging/dosbox-staging/issues/262#issuecomment-706675838

You can see how we're initializing libfluidsynth here: https://github.com/dosbox-staging/dosbox-staging/blob/master/src/midi/midi_fluidsynth.cpp with the sf2 load on line 94.

We don't touch the synth.dynamic-sample-loading setting. Does this default to true, and we should set it to false prior to the sf2 load?

[mawe42]

Well, reading the comment in https://github.com/dosbox-staging/dosbox-staging/issues/262#issuecomment-706675838 it looks like the problem disappears when HoMM2 (whatever that is) is moved to tmpfs. According to the comment, the Soundfont is always located on the hdd. So it's not FluidSynth's IO that is the problem here, but whatever happens with "HoMM2".

I would guess that your anlysis in https://github.com/dosbox-staging/dosbox-staging/issues/262#issuecomment-705929962 is correct: you are calling FluidSynth in your main rendering loop, your buffer size is too small and you get buffer underruns in your mixer.

[mawe42]

We don't touch the synth.dynamic-sample-loading setting. Does this default to true, and we should set it to false prior to the sf2 load?

It defaults to false.

[mawe42]

Maybe one more clarification, because there seems to be some confusion about FluidSynths threading behaviour: You are not using our audio drivers but call the rendering function directly (e.g. fluid_synth_write_s16). The rendering functions are intended to be called from the "synthesis thread", i.e. a thread that runs at high priority and ideally with very little other duties other than calling FS rendering func and feeding the output to the audio output buffers.

When you use our audio drivers, those threads are automatically created for you. But as you handle rendering and mixing yourself, you have to make sure your are calling the render function from a suitable thread (and with a suitable interval and buffer size).

[dreamer]

Maybe one more clarification, because there seems to be some confusion about FluidSynths threading behaviour: You are not using our audio drivers but call the rendering function directly (e.g. fluid_synth_write_s16). The rendering functions are intended to be called from the "synthesis thread", i.e. a thread that runs at high priority and ideally with very little other duties other than calling FS rendering func and feeding the output to the audio output buffers.

When you use our audio drivers, those threads are automatically created for you. But as you handle rendering and mixing yourself, you have to make sure your are calling the render function from a suitable thread (and with a suitable interval and buffer size).

Thank you for this confirmation! That's how we expected FS to work from the very beginning, but some commenters insisted that's not the case based on synth.cpu-cores description. Does this setting affect our usecase in any way (I guess it does not)?

[mawe42]

synth.cpu-cores controls how many "extra" threads do the actual rendering work that is then returned synchronously by the render function. So for example, if you set cpu-cores to 4 and then call fluid_synth_write_s16, that function will attempt to split the work it needs to do between it's own calling thread and three additional (internal) worker threads. As soon as all threads have done their work, their results are collected and the resulting buffer is returned to the caller.

So no, it doesn't really affect how you need to integrate FS into dosbox.

Edit, to be more precise: the extra threads are not actually joined, they live as long as the synth. But the render function waits for all extra threads to have finished processing their data, collects everything and then returns.

[kcgen]

@mawe42 , @derselbst -- thanks for the fast and detailed feedback, as well as elaborating on what's happening under the hood regarding the synth threading.

I also had concerns that the SF2 load process might be asynchronous, however it indeed blocks during the load (which is great).

Thanks again -- and keep up the excellent work; FluidSynth keeps getting better :rocket:

(closing)