search

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
Apache License 2.0
31 stars 10 forks source link

Support SGLang & modify model source API #68

Closed kerthcet closed 3 months ago

kerthcet commented 3 months ago

What this PR does / why we need it

  1. Support SGLang
  2. Support loading models from ModelScope
  3. Change model dataSource to Source
  4. Support model revision when loading from modelHub
  5. Update the examples
  6. Update README.md

Which issue(s) this PR fixes

Fixes https://github.com/InftyAI/llmaz/issues/39 Fixes https://github.com/InftyAI/llmaz/issues/19

Special notes for your reviewer

Does this PR introduce a user-facing change?

None
kerthcet commented 3 months ago

/lgtm /approve

kerthcet commented 3 months ago

/kind feature /api-change

kerthcet commented 3 months ago

/lgtm