Parallel SF3 sample decoding

[derselbst]

Related discussion on the mailing list

https://lists.nongnu.org/archive/html/fluid-dev/2020-10/msg00054.html

Is your feature request related to a problem?

Loading of SF3 files makes poor use of available CPU power. It should be accelerated by parallel decoding the OGG Vorbis compressed samples.

Describe the solution you'd like

Parallelization using openMP tasks: https://github.com/FluidSynth/fluidsynth/tree/parallelize-sf3-loading

Additional context

Currently, there is a race condition on the soundfont's FILE*. A possible way to remove this limitation would be to mmap() the soundfont file. That way we would have full control over offset in the file. Drawback: Not all filesystems allow mmap()ing files. So we still need to provide a fallback to the old FILE* method.

Alternative would be to completely rewrite the soundfont loader. Since the loader is exposed via public API, this would also cause much pain downstream. Pain that IMO isn't justified by this enhancement feature.

Yet, it would be good to complete this feature before releasing 2.2.0. This is because 2.2.0 will have an SOVERSION bump. That way any necessary minor API adjustments could be considered as well.

[mawe42]

A possible way to remove this limitation would be to mmap() the soundfont file. That way we would have full control over offset in the file. Drawback: Not all filesystems allow mmap()ing files. So we still need to provide a fallback to the old FILE* method.

mmap'ing the file sounds good, in my opinion. Maybe using glibs g_mapped_file_new?

And is mmap'ing really dependent on the file system? I thought the problem would only be platforms without MMU, and maybe a size restriction in 32-bit platforms?

Alternative would be to completely rewrite the soundfont loader. Since the loader is exposed via public API, this would also cause much pain downstream. Pain that IMO isn't justified by this enhancement feature.

I agree.

[derselbst]

And is mmap'ing really dependent on the file system?

I was referring to https://man7.org/linux/man-pages/man2/mmap.2.html

ENODEV The underlying filesystem of the specified file does not support memory mapping.

But, yes, what you said probably also applies. g_mapped_file_new looks good!

[mawe42]

I was referring to https://man7.org/linux/man-pages/man2/mmap.2.html

I really wonder which real-world filesystems do not support memory mapping. My guess is that it mostly (or only?) affects special filesystems like /sys or special files like /dev/null. Very old Linux kernels had lacking mmap support for vfat, but that was at least 10 years ago, probably much longer. Even nfs supports memory mapping.

And the documentation for CreateFileMapping on Windows does not mention anything about filesystem type restrictions: https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-createfilemappinga

OS/2 doesn't have mmap support, not sure how glib handles that with g_mapped_file_new. But OS/2 support has been dropped from glib in 2013 anyway. If we still want to support OS/2, maybe we could limit the mmap to SF3 files and don't support SF3 on OS/2?

[derselbst]

maybe we could limit the mmap to SF3 files and don't support SF3 on OS/2?

For those cases we'll need a fallback to FILE* anyway, i.e. whenever g_mapped_file_new returns NULL.