search

rentruewang / koila

Prevent PyTorch's `CUDA error: out of memory` in just 1 line of code.
https://koila.rentruewang.com
MIT License
1.82k stars 63 forks source link

Integration with huggingface #12

Open arsalan993 opened 2 years ago

arsalan993 commented 2 years ago

If somehow we can integrate this with hugging face models while doing inference then its job is done for production-level deployments.

rentruewang commented 2 years ago

Agreed. Will work on it when I have the time.