snowflakedb / snowflake-ml-python

Apache License 2.0
38 stars 8 forks source link

ValueError: DataFrame.dtypes for data must be int, float or bool. #41

Closed ChuliangXiao closed 5 months ago

ChuliangXiao commented 11 months ago

Try to use LGBMClassifier

from snowflake.ml.modeling.lightgbm import LGBMClassifier

clf = LGBMClassifier(
    input_cols=INPUT_COLUMS,
    label_cols=LABEL_COLUMN,
    output_cols=OUTPUT_COLUMN,
    **params
)
clf.fit(train_df)

result = clf.predict(test_df)

result.groupBy(OUTPUT_COLUMN).count().show()

and got the following error

raise ValueError("DataFrame.dtypes for data must be int, float or bool.\n"
ValueError: DataFrame.dtypes for data must be int, float or bool.
Did not expect the data types in the following fields: ... some fields...

while all those fields are DoubleType() or LongType(). Still got the same issue after .cast(FloatType()).

sfc-gh-xjiang commented 11 months ago

Hi @ChuliangXiao, this error may come from LightGBM model itself. Would you mind looking at solutions like this or this? If they don't work for you, could you provide any more details for your problem, such as the schema of your table - which would be easier for me to reproduce your error?

ChuliangXiao commented 11 months ago

Hi @ChuliangXiao, this error may come from LightGBM model itself. Would you mind looking at solutions like this or this? If they don't work for you, could you provide any more details for your problem, such as the schema of your table - which would be easier for me to reproduce your error?

Had the same issue with XGBoost. It's fine while manually creating a stored procedure in which Snowflake DF is converted .toPandas() to Pandas DF

sfc-gh-xjiang commented 8 months ago

Hey @ChuliangXiao , how is everything going? I want to follow up the issue and see if you can provide a dataset/table schema so that I can reproduce the issue. If the problem is solved, I can close this issue.

sfc-gh-wzhao commented 5 months ago

Hi @ChuliangXiao , we are closing this stale issue. If you have any updates, please re-open it. Thank you for supporting.