auto_lightgbm not working

jarichb commented 3 months ago

Describe the bug auto_lightgbm is not working, when following the documentation: https://docs.functime.ai/forecasting/#quantile-regression

To Reproduce

import json
from timeit import default_timer

import polars as pl

from functime.cross_validation import train_test_split
from functime.forecasting import auto_linear_model, linear_model, naive, snaive, auto_lightgbm
from functime.metrics import smape
from functime.preprocessing import scale
from functime.seasonality import add_fourier_terms

start_time = default_timer()

y = pl.read_parquet(
    "https://github.com/functime-org/functime/raw/main/data/commodities.parquet"
)
entity_col, time_col = y.columns[:2]
X = y.select([entity_col, time_col]).pipe(add_fourier_terms(sp=12, K=6)).collect()

print("🎯 Target variable (y):\n", y)
print("📉 Exogenous variables (X):\n", X)

test_size = 3
freq = "1mo"
y_train, y_test = train_test_split(test_size)(y)
X_train, X_test = train_test_split(test_size)(X)

y_pred_10 = auto_lightgbm(alpha=0.1, freq="1d")(y=y_train, fh=28)
y_pred_90 = auto_lightgbm(alpha=0.9, freq="1d")(y=y_train, fh=28)

Expected behavior that the example in the documentation would at least execute

Screenshots

Desktop (please complete the following information):

OS: Mac
Functime version: 0.9.5.

Additional context As per requested on LinkedIn

baggiponte commented 2 months ago

Ciao @jarichb, thanks for reporting and sorry for the late reply. I see the error! Did you make sure to install functime[lgb]?

We documented that on this page: how can we improve that?

I think I need to make this error be raised sooner.

jarichb commented 2 months ago

interestingly enough, I can't seem to get it installed like that either, see error below. It does mention requirement already satisfied but it still can't do the above function, so it seems like it doesn't install it properly.

UPDATE: this error was resolved with 'brew install libomp', and is a LGB specific thing, now I got it to work! I do think that 'ImportError' not callable is a rather vague error though, if it could display to make the specific install, I feel like that would help.

(.env) jarich@Jarichs-Air functime % pip install "functime[lgb]"

Requirement already satisfied: functime[lgb] in ./.env/lib/python3.8/site-packages (0.9.5)
Requirement already satisfied: cloudpickle in ./.env/lib/python3.8/site-packages (from functime[lgb]) (3.0.0)
Requirement already satisfied: flaml<3,>=2.0.2 in ./.env/lib/python3.8/site-packages (from functime[lgb]) (2.1.2)
Requirement already satisfied: holidays in ./.env/lib/python3.8/site-packages (from functime[lgb]) (0.46)
Requirement already satisfied: numpy in ./.env/lib/python3.8/site-packages (from functime[lgb]) (1.24.4)
Requirement already satisfied: polars>=0.20.7 in ./.env/lib/python3.8/site-packages (from functime[lgb]) (0.20.21)
Requirement already satisfied: scikit-learn<2,>=1.2.2 in ./.env/lib/python3.8/site-packages (from functime[lgb]) (1.3.2)
Requirement already satisfied: scipy in ./.env/lib/python3.8/site-packages (from functime[lgb]) (1.10.1)
Requirement already satisfied: tqdm in ./.env/lib/python3.8/site-packages (from functime[lgb]) (4.66.2)
Requirement already satisfied: typing-extensions in ./.env/lib/python3.8/site-packages (from functime[lgb]) (4.11.0)
Collecting lightgbm (from functime[lgb])
  Using cached lightgbm-4.3.0.tar.gz (1.7 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: joblib>=1.1.1 in ./.env/lib/python3.8/site-packages (from scikit-learn<2,>=1.2.2->functime[lgb]) (1.4.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in ./.env/lib/python3.8/site-packages (from scikit-learn<2,>=1.2.2->functime[lgb]) (3.4.0)
Requirement already satisfied: python-dateutil in ./.env/lib/python3.8/site-packages (from holidays->functime[lgb]) (2.9.0.post0)
Requirement already satisfied: six>=1.5 in ./.env/lib/python3.8/site-packages (from python-dateutil->holidays->functime[lgb]) (1.16.0)
Building wheels for collected packages: lightgbm
  Building wheel for lightgbm (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for lightgbm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [40 lines of output]
      2024-04-16 21:20:24,534 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/cmake/data/bin/cmake --version
      2024-04-16 21:20:24,540 - scikit_build_core - INFO - CMake version: 3.29.2
      *** scikit-build-core 0.8.2 using CMake 3.29.2 (wheel)
      2024-04-16 21:20:24,545 - scikit_build_core - INFO - Build directory: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build
      *** Configuring CMake...
      2024-04-16 21:20:24,555 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/ninja/data/bin/ninja --version
      2024-04-16 21:20:24,646 - scikit_build_core - INFO - Ninja version: 1.11.1
      2024-04-16 21:20:24,647 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/cmake/data/bin/cmake -S. -B/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build -DCMAKE_BUILD_TYPE:STRING=Release -C/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build/CMakeInit.txt -DCMAKE_MAKE_PROGRAM=/private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/ninja/data/bin/ninja -D__BUILD_FOR_PYTHON:BOOL=ON
      loading initial cache file /var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build/CMakeInit.txt
      -- The C compiler identification is AppleClang 12.0.5.12050022
      -- The CXX compiler identification is AppleClang 12.0.5.12050022
      -- Detecting C compiler ABI info
      -- Detecting C compiler ABI info - done
      -- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
      -- Detecting C compile features
      -- Detecting C compile features - done
      -- Detecting CXX compiler ABI info
      -- Detecting CXX compiler ABI info - done
      -- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
      -- Detecting CXX compile features
      -- Detecting CXX compile features - done
      -- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
      -- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES)
      -- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND)
      -- Found OpenMP_C: -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include
      -- Found OpenMP_CXX: -Xpreprocessor -fopenmp -I/opt/homebrew/opt/libomp/include
      -- Found OpenMP: TRUE
      -- Performing Test MM_PREFETCH
      -- Performing Test MM_PREFETCH - Failed
      -- Performing Test MM_MALLOC
      -- Performing Test MM_MALLOC - Success
      -- Using _mm_malloc
      -- Configuring done (0.7s)
      -- Generating done (0.0s)
      -- Build files have been written to: /var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build
      *** Building project with Ninja...
      2024-04-16 21:20:25,354 - scikit_build_core - INFO - RUN: /private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-build-env-fooxd38o/normal/lib/python3.8/site-packages/cmake/data/bin/cmake --build /var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/tmpah0f76ye/build
      ninja: error: '/opt/homebrew/opt/libomp/lib/libomp.dylib', needed by '/private/var/folders/fl/qtsnzd1x0k3dn109zhspr_yc0000gn/T/pip-install-9ka6inly/lightgbm_51586f541a1746beb85cf9098a705212/lib_lightgbm.so', missing and no known rule to make it

      *** CMake build failed
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for lightgbm
Failed to build lightgbm
ERROR: Could not build wheels for lightgbm, which is required to install pyproject.toml-based projects

baggiponte commented 2 months ago

UPDATE: this error was resolved with 'brew install libomp', and is a LGB specific thing, now I got it to work! I do think that 'ImportError' not callable is a rather vague error though, if it could display to make the specific install, I feel like that would help.

Yes, I should explicitly mark that lightgbm requires libomp. Will add it to the docs and then proceed to close the issue. Does this sound right to you? 😊

jarichb commented 2 months ago

Yes, sounds good to me! Thank you

On Wed, 17 Apr 2024, 17:49 baggiponte, @.***> wrote:

UPDATE: this error was resolved with 'brew install libomp', and is a LGB specific thing, now I got it to work! I do think that 'ImportError' not callable is a rather vague error though, if it could display to make the specific install, I feel like that would help.

Yes, I should explicitly mark that lightgbm requires libomp. Will add it to the docs and then proceed to close the issue. Does this sound right to you? 😊

— Reply to this email directly, view it on GitHub https://github.com/functime-org/functime/issues/190#issuecomment-2061617911, or unsubscribe https://github.com/notifications/unsubscribe-auth/AP4NCDU56APNWAC2XE7XJ5DY52KXBAVCNFSM6AAAAABF5IZGVWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRRGYYTOOJRGE . You are receiving this because you were mentioned.Message ID: @.***>

baggiponte commented 2 months ago

Ciao @jarichb! The preview of the new docs is here: I'll merge the request as soon as you give it a green light 😊

functime-org / functime

auto_lightgbm not working #190