Investigate -fno-semantic-interposition build option

[rth]

Fedora 32 has a proposal to build python with with -fno-semantic-interposition for better performance (https://fedoraproject.org/wiki/Changes/PythonNoSemanticInterpositionSpeedup),

We add the -fno-semantic-interposition compiler/linker flag when building Python interpreters, as it provides significant performance improvement, up to 27% depending on the workload. Users will no longer be able to use LD_PRELOAD to override a symbol from libpython, which we consider a good trade off for the speedup.

It would be interesting to investigate whether the same could be done for conda-forge Linux (and possibly MacOS) builds.

[jakirkham]

cc @jjhelmus @mingwandroid (as you may have thoughts here 😉)

[mingwandroid]

Losing LD_PRELOAD is unfortunate. We could test it I suppose though.

[jjhelmus]

The current python conda packages are linked statically against libpython. I do not believe there would be any performance improvements from building with -fno-semantic-interposition since there is no dynamic libpython library to interpose.

Fedora dynamically links the python interpreter to libpython and therefore sees gains from building with -fno-semantic-interposition. The original Fedora proposal began with a discussion to statically link the interpreter to achieve performance gains.

[mingwandroid]

You can still interpose glibc.

[mingwandroid]

I expect the perf. gains in disabling that would be minimal though.

[jjhelmus]

According to the Fedora proposal it is still possible to interpose glibc and other libraries. Interposition is only disabled for libpython.

""" It is still possible to use LD_PRELOAD to override symbols in other libraries (for example in glibc). """

[mingwandroid]

OK then I don't think there's any point here, happy to be benchmarked wrong though.

[rth]

Thanks for investigating! Looks like this issue can be closed then?

[mingwandroid]

We do build a libpython dynamic library. We could still enable it for that if anyone cares?