Make `DEFAULT_HF_HUB_MODEL_EXPORT_DIRECTORY` configurable through environment variable

aws / sagemaker-huggingface-inference-toolkit

Apache License 2.0

240 stars 60 forks source link

Make `DEFAULT_HF_HUB_MODEL_EXPORT_DIRECTORY` configurable through environment variable #78

Closed philschmid closed 1 year ago

philschmid commented 1 year ago

Currently, DEFAULT_HF_HUB_MODEL_EXPORT_DIRECTORY points to /.sagemaker/mms/models, which is only 50GB, where ~27GB are already reserved with system things. This means that customers can only deploy models with ~23GB of size. We should either change this by default to /tmp/sagemaker/mms/models or make it configurable through an environment variable.

philschmid commented 1 year ago

Not needed in the latest version since the model is not longer copied out of /opt/ml/model