openMP conflict with numba

[bcharlier]

On macOS numba and pykeops (or even torch !) conflict due a static link problem:

from numba import njit, prange
import numpy as np

from pykeops.numpy import LazyTensor

@njit(parallel=True)
def ompbug(n):
    j = 0
    a = np.zeros(n)
    for i in prange(n):
        a[i] = 1
    return a.sum()

 print(ompbug(100))

gives the error

[KeOps] Warning : Cuda libraries were not detected on the system ; using cpu only mode
OMP: Error #15: Initializing libiomp5.dylib, but found libiomp5.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

[lyounes]

This issue happened on macOS 13.4.1 using intel python.

The problem seems to come from torch (and pykeops importing torch if it's available) rather than pykeops. Uninstalling torch fixes the issue but this only works if torch is not used elsewhere (which was true for me).

See https://github.com/pytorch/pytorch/issues/44282

[jjerphan]

This condition is documented in threadpoolctl.

A good way to prevent such an issue is to use conda packages rather than python wheels.

For full background, see pypackaging-native, more especially in the Native Dependencies section.