[BUG]: Regression Compare Models only returning LightGBM and Dummy

isaiahxcruz commented 2 years ago

pycaret version checks

[X] I have checked that this issue has not already been reported here.
[X] I have confirmed this bug exists on the latest version of pycaret.
[ ] I have confirmed this bug exists on the master branch of pycaret (pip install -U git+https://github.com/pycaret/pycaret.git@master).

Issue Description

I recently installed the upgraded pycaret via pip3. I created a time series with random values for y and attempted to run a regression experiment. When it runs, it goes through a list of all the models provided but the result is a return of LightGBM and Dummy regressor, nothing else.

Reproducible Example

import pandas as pd
import numpy as np

dates = pd.date_range(start='1/1/2021', end='1/1/2022', freq='D')
vcores = np.random.randint(1, 50, size=len(dates))
data={'dates':dates, 'vcores':vcores}
df = pd.DataFrame(data=data)
df.set_index(dates, inplace=True)
df.drop('dates', axis=1, inplace=True)

train_ind=int(len(df)*0.8)
train=df[:train_ind]
test=df[train_ind:]

from pycaret.regression import *

exp = setup(train, train_size=0.8, target='vcores', data_split_shuffle=False, fold_strategy='timeseries')
compare_models()

Expected Behavior

huber | Huber Regressor | 0.0494 | 0.0130 | 0.0713 | 0.9268 | 0.0147 | 0.0127 | 0.0290 -- | -- | -- | -- | -- | -- | -- | -- | -- Lasso Regression | 0.0653 | 0.0335 | 0.1148 | 0.8545 | 0.0255 | 0.0196 | 0.0070 Elastic Net | 0.0652 | 0.0335 | 0.1148 | 0.8544 | 0.0255 | 0.0196 | 0.0070 Gradient Boosting Regressor | 0.2910 | 0.3991 | 0.3570 | -0.4301 | 0.0685 | 0.0863 | 0.1210 Extra Trees Regressor | 0.3423 | 0.5543 | 0.4055 | -1.0313 | 0.0777 | 0.1036 | 0.5070 Orthogonal Matching Pursuit | 0.1437 | 0.5384 | 0.3034 | -1.0484 | 0.0181 | 0.0469 | 0.0060 AdaBoost Regressor | 0.3493 | 0.5595 | 0.4185 | -1.0499 | 0.0792 | 0.1040 | 0.0890 Random Forest Regressor | 0.3658 | 0.5649 | 0.4228 | -1.0551 | 0.0815 | 0.1099 | 0.4960 Decision Tree Regressor | 0.3968 | 0.6173 | 0.4948 | -1.4153 | 0.0901 | 0.1118 | 0.0080 Extreme Gradient Boosting | 0.4873 | 0.8058 | 0.5700 | -2.3462 | 0.1048 | 0.1401 | 0.3240 CatBoost Regressor | 0.5834 | 0.9216 | 0.6547 | -2.4012 | 0.1247 | 0.1731 | 1.9830 Light Gradient Boosting Machine | 0.5174 | 0.9572 | 0.5895 | -2.4582 | 0.1107 | 0.1528 | 0.0230 Passive Aggressive Regressor | 0.9121 | 1.5765 | 0.9701 | -5.6274 | 0.1817 | 0.2434 | 0.0080 K Neighbors Regressor | 0.7623 | 1.1585 | 0.8794 | -7.0430 | 0.1613 | 0.2138 | 0.3310 Lasso Least Angle Regression | 1.3113 | 2.9353 | 1.4034 | -11.8521 | 0.2544 | 0.3879 | 0.0050 Linear Regression | 4.0729 | 145.1286 | 5.7263 | -483.7128 | 0.3858 | 0.9488 | 0.0060 Bayesian Ridge | 7.3757 | 2826.3201 | 17.6026 | -10669.0481 | 0.3243 | 2.7772 | 0.0320 Ridge Regression | 8.3168 | 4331.3753 | 20.8806 | -16361.0021 | 0.2116 | 3.1673 | 0.0050 Least Angle Regression | 18583978932258.3008 | 5903475984839208765730324480.0000 | 24454644538648.7266 | -8865067586334214311708196864.0000 | 10.0498 | 5121901449923.7939 | 0.0160

Actual Results

lightgbm    Light Gradient Boosting Machine 12.2235 203.2234    14.1625 -0.0368 0.7676  1.4719  0.0140
dummy   Dummy Regressor 12.2235 203.2234    14.1625 -0.0368 0.7676  1.4719  0.0060

Installed Versions

2.3.10

joncassidy commented 2 years ago

I have the same issue when adding many (100+) float16 features, but the issue does not happen if the features are float instead.

celestinoxp commented 2 years ago

Hi, are you using pycaret 2.3.10? i think it's problem from scikit-learn, because pycaret uses "old" 0.24.2 version of scikit-learn. Scikit-learn have a lot of bugfixes since 0.24.2. I think i have seen something like there in scikit bugfixes...

Suggestion: install latest pycaret (rc3 at this time) and it will install latest scikit-learn. Test if works. After that tell me if works.

Note: To avoid problems with packages uninstall first pycaret and sckit-learn, then install latest pycaret (it will install scikit-learn automatically)

if you don't know how to do it, use these commands:

pip uninstall pycaret -y pip uninstall scikit-learn -y pip install --pre pycaret or pip install --pre pycaret[full]

Important: if you don't use python 3.8 or python 3.9 I recommend creating a new environment with one of these versions.

moezali1 commented 1 year ago

By changing few things in setup I was able to run your code successfully using 3.0rc4

pip install --pre pycaret

Check out the Notebook.

pycaret / pycaret