search

snowflakedb / snowflake-ml-python

Apache License 2.0
37 stars 7 forks source link

SNOW-992407: Use FileSet module without additional requirements #78

Open kylejcaron opened 7 months ago

kylejcaron commented 7 months ago

I'm finding the FileSet API really useful for jax/numpyro/pyro model development since I can use a torch dataloader and directly interface with snowflake without having to save anything locally. I'd love to not have to install pandas, sklearn, xgboost, lightgbm, and tensorflow in order to use the fileset api.

It would be great if this could have separate requirements, or be separated out, or have some of the clunky package requirements like those mentioned above be made optional

sfc-gh-wzhao commented 7 months ago

Hi kylecaron,

Currently pandas, sklearn and xgboost are hard requirements, but all others are extra requirements that means unless installing via pip install snowflake-ml-python[lightgbm], pip install snowflake-ml-python[tensorflow] or pip install snowflake-ml-python[all], they won't be installed.

You could use pip install snowflake-ml-python or conda install snowflake-ml-python if you don't want to install these extra dependencies. For details, please refer to our docs at https://docs.snowflake.com/en/developer-guide/snowpark-ml/index#installing-snowpark-ml .

sfc-gh-sdas commented 3 weeks ago

are you still working on it? Can you work with dependencies on sklearn and xgboost?