[BUG] Intel OpenMP (libiomp) and LLVM OpenMP (libomp) loaded at same time

[sharmuz]

Describe the bug When darts.models.forecasting.croston is run it imports statsforecast.core, within which threadpoolctl.ThreadpoolController() is created. This then detects that both Intel libiomp and LLVM libomp are both loaded. This throws a warning due to the issue outlined here.

"Therefore this problem would only happen if some packagers decide to start shipping Python packages built with LLVM/Clang instead of GCC"

Note that if statsforecast.core is imported outside of darts, no such warning is thrown. So it seems that a dependency of darts is built with LLVM. Any idea what's at fault?

/Users/<snip>/.venv/lib/python3.10/site-packages/threadpoolctl.py:1214: RuntimeWarning: 
Found Intel OpenMP ('libiomp') and LLVM OpenMP ('libomp') loaded at
the same time. Both libraries are known to be incompatible and this
can cause random crashes or deadlocks on Linux when loaded in the
same Python program.
Using threadpoolctl may cause crashes or deadlocks. For more
information and possible workarounds, please see
    https://github.com/joblib/threadpoolctl/blob/master/multiple_openmp.md

  warnings.warn(msg, RuntimeWarning)

To Reproduce

1. Be on macos
2. Install darts (from pypi)
3. import darts.models.forecasting.croston

Expected behavior No warning.

System (please complete the following information):

• Python version: 3.10.13 (also tested with 3.10.14, 3.10.15, 3.11.10)
• darts version: 0.30.0 (also tested with 0.29.0, with numpy==1.26.4
• OS version: macOS Sonoma 14.6.1

Additional context Add any other context about the problem here.

[madtoinou]

Hi @sharmuz,

Thank you for reporting this, the culprit is probably lightgbm or xbgboost. We'll try to investigate this issue asap.

[dennisbader]

Hi @sharmuz, are you running on a Mac with an M chip? If so, some of the dependencies installed with PyPI might not use the correct dependencies for the M-chip (e.g. they install the intel version). For me this is fixed when installing the following dependencies with conda before installing darts:

conda install catboost pytorch lightgbm -y
pip install darts

[sharmuz]

@dennisbader I'm on a 2020 Intel Mac.

Also I'm using uv for package management. I could try having it install lightgbm from source using gcc. This is the option FYI

[madtoinou]

@sharmuz, a little bit of follow-up; did the no-binary-package option of uv for darts dependencies solved the issue?

[sharmuz]

@madtoinou thanks for the nudge - I gave it a go, but lightgbm is failing to build using gcc. I created a new issue:

https://github.com/microsoft/LightGBM/issues/6720