aws / sagemaker-huggingface-inference-toolkit

Apache License 2.0
240 stars 60 forks source link

feature: Serve model directly from original model artifact directory … #71

Closed davidthomas426 closed 2 years ago

davidthomas426 commented 2 years ago

…to save disk space.

Issue #, if available:

Description of changes: Updated code and tests so that, except in HF Hub use case where the model artifact is fetched at runtime, the original model is served directly from the original model artifact location rather than calling model-archiver. This saves disk space, which matters especially for large models and in cases where the relevant disk volume did not have much space.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.