Add support for multithread when downloading weights

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes

Apache License 2.0

25 stars 10 forks source link

Add support for multithread when downloading weights #41

Closed kerthcet closed 3 months ago

kerthcet commented 3 months ago

We should test the improvement once we support OCI registry

Discard the approach of rust language right now because the limitation is due to the cloud vendor NAT, like 200Mpbs ~= 25 MB/s. https://github.com/InftyAI/llmaz/pull/40

kerthcet commented 3 months ago

/lgtm /approve

kerthcet commented 3 months ago

/kind feature