Add model caching for Stable Diffusion

ratnopamc commented 6 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

What is the outcome that you are trying to reach?

Currently the Stable diffusion model is directly fetched from HuggingFace and adds to the latency of the inference. It would be good to download the model and load it from a storage like S3.

Describe the solution you would like

Update the instructions to download the model from Hugging Face. Store the model in S3 or any other storage. Load the model from the storage during inference.

Describe alternatives you have considered

Additional context

lindarr915 commented 1 month ago

I would like to work on this

lindarr915 commented 1 month ago

Using S3 mountpoints and NVMe local storage for storing ML models, it can save cold start times

ratnopamc commented 1 month ago

Thanks @lindarr915; assigning to you. Looking forward to the PR!

awslabs / data-on-eks