Feat: support modelscope

InftyAI / llmaz

☸️ Easy, advanced inference platform for large language models on Kubernetes

Apache License 2.0

25 stars 10 forks source link

Feat: support modelscope #45

Closed vicoooo26 closed 3 months ago

vicoooo26 commented 3 months ago

Support loading model from Modelscope using Modelscope Python SDK

Ref https://github.com/InftyAI/llmaz/issues/19

kerthcet commented 3 months ago

Hey bro, thank you very much for your contribution, we're at very early stage so some baselines haven't been established like the tests, I'll try my best this weekend to build it.

kerthcet commented 3 months ago

Please add an example under /docs/examples for reference, and have you tried this?

kerthcet commented 3 months ago

One more thing, can you add an integration test for Defaulting test at /test/integration/webhoook/model_test.go to make sure configuring modelScope is allowed? We currently have a test for huggingface there.

kerthcet commented 3 months ago

/lgtm /approve

Thanks @vicoooo26