search

FFTW / fftw3

DO NOT CHECK OUT THESE FILES FROM GITHUB UNLESS YOU KNOW WHAT YOU ARE DOING. (See below.)
GNU General Public License v2.0
2.73k stars 665 forks source link

Thread Safe Planner #16

Closed x42 closed 8 years ago

x42 commented 10 years ago

http://www.fftw.org/fftw3_doc/Thread-safety.html mentions

We do not think this should be an important restriction

I hope I can convince you otherwise.

Problem Description

fftw3 is used by a variety of audio plugins.

Those plugins are loaded into the host's memory-space (usually an audio workstation). The host has limited control of what the plugin does internally, and the plugins do not know about each other.

There is no way to ensure that two independent plugins which are linked against libfftw do not run the shared planner simultaneously. Nor is there a possibility to control this on host application level.

When two independent plugins create fftw plans the application usually segfaults or similar undesired effects manifest.

Possible solutions for this include:

  1. Statically link plugins against libfftw. Every plugin will have its own copy. The plans are not shared with other plugins (which is mostly fine). This still requires a bit of special attention: (fftw symbol visibility needs to be overridden for static links and the plugin must protect its planning routings for multiple instances of itself). Furthermore distributors must honor that (special built of fftw + static link). -- It is very unlikely that both plugin-authors and various gnu/linux-distributors do get this right (most distros dislike static linking) for the growing number of audio-plugins using fftw.
  2. process separate all plugins in the host. That is not a viable option for Digital Audio Workstations where low-latency is important, context switches (particularly realtime thread) heavy and inter-process communication does not scale (compared to shared memory), especially so if the DAW does not limit audio track or channel count.
  3. Discourage use of fftw for audio-plugins or even refuse to load plugis using it in the host. -- not the best idea :)
  4. Ship a special (ABI compatible) build of libfftw with the host application which protects the planner. Plugins in the same memory space will use the already loaded library. This requires patching libfftw, but when doing so... why not do it upstream directly. Otherwise it has similar issues as (1).

    Discussion

The issue at hand is not limited to audio-application, there are likely other applications with similar problems out there (gnu-octave comes to mind, but I don't know for certain).

As the Thread-safety page mentions, it's as simple as

wrap a semaphore lock around any calls to the planner

Is there some good reason why libfftw does not do this by default?

Existing applications should not be affected by this (they're not supposed to call the planner from different threads), but that change would make all the difference for multi-threaded plugin hosts.

I suppose it could be a bit of work to wrap all planner entry-points with a semaphore, yet there may be a neat simple solution using #define.

I'll be happy to look into this, but before going that way, I'd like to ask if such a change would be accepted by fftw or if there is an even better solution planned for future version that will make fftw's planner thread-safe.

yours truly, robin - for the linux-audio community and for himself

Notable audio plugins using fftw3: http://calf.sourceforge.net/ http://factorial.hu/plugins/lv2/ir http://guitarix.sourceforge.net/ http://breakfastquay.com/rubberband/ http://plugin.org.uk/ http://zynaddsubfx.sourceforge.net/ https://github.com/x42/meters.lv2 ...

Notable affected plugin hosts: http://ardour.org/ http://qtractor.sourceforge.net/ https://github.com/falkTX/Carla/ ...

see also https://community.ardour.org/node/8271

rlcamp commented 7 years ago

I think there is still room for improvement here. Perhaps an FFTW_NO_GLOBAL_STATE planning flag? I'm willing to wait a hair longer for an FFTW_ESTIMATE if it means planning can happen indiscriminately in independently-developed plugins without counting on libfftw3f_threads being available.

Also: in light of the fftw_destroy_plan bug fixed in 3.3.6, and the fact that 3.3.5 is still the newest version in some major repositories: is there a way to check, at compile time, whether 3.3.5 or 3.3.6 is being linked against? There is the fftwf_version macro, but it's a string, you can't test an inequality against it in the C preprocessor.