fix:load models cost seconds

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!

Apache License 2.0

30 stars 10 forks source link

fix:load models cost seconds #175

Closed qinguoyi closed 1 month ago

qinguoyi commented 1 month ago

What this PR does / why we need it

datetime.now() returns a datetime object containing date and time, not simple seconds or milliseconds

Which issue(s) this PR fixes

None

Special notes for your reviewer

Does this PR introduce a user-facing change?

None

kerthcet commented 1 month ago

/kind bug /lgtm /approve

Thanks!

By the way, I'm developing a new project https://github.com/InftyAI/Manta which tries to share model weights across the cluster, which means we may change part of the code in the future. Just FYI.