search

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes
Apache License 2.0
23 stars 10 forks source link

Concurrently download the main container image when downloading weights #50

Open kerthcet opened 3 months ago

kerthcet commented 3 months ago

This can help to optimize the startup time of the Pod, however, usually this is limited by the bandwidth, which means it will slow down the weight downloading. If the image has been downloaded, there's no different since it's cached.

But if your registry is deployed in the intranet, it will still benefit your startup time.

kerthcet commented 3 months ago

/kind question

kerthcet commented 3 months ago

/kind feature /remove-kind question

kerthcet commented 3 months ago

/priority important-longterm