search

aws / aws-step-functions-data-science-sdk-python

Step Functions Data Science SDK for building machine learning (ML) workflows and pipelines on AWS
Apache License 2.0
285 stars 87 forks source link

Make sagemaker an feature install #116

Closed milesgranger closed 2 years ago

milesgranger commented 3 years ago

pip install stepfunctions[sagemaker]

Why? :thinking:

Because sagemaker comes with some not-so-light dependencies. And in situations where the sagemaker module isn't used, we're taking up some valuable space.

ie. After installing we have things like scipy, numpy which aren't needed if one is not using the sagemaker module.

✦ ❯ du -h --max-depth=1  venv/lib/python3.7/site-packages/ | sort -hr
240M    venv/lib/python3.7/site-packages/
65M venv/lib/python3.7/site-packages/scipy
56M venv/lib/python3.7/site-packages/botocore
33M venv/lib/python3.7/site-packages/numpy.libs
30M venv/lib/python3.7/site-packages/scipy.libs
28M venv/lib/python3.7/site-packages/numpy
wong-a commented 2 years ago

Resolving in favour of https://github.com/aws/aws-step-functions-data-science-sdk-python/issues/184