Closed kerthcet closed 3 weeks ago
Now we can serving GGUF models with llama.cpp, what's more, we can run e2e tests with cpu.
Fixes https://github.com/InftyAI/llmaz/issues/65
Support llama.cpp as another backend
/kind feature
/lgtm /approve
What this PR does / why we need it
Now we can serving GGUF models with llama.cpp, what's more, we can run e2e tests with cpu.
Which issue(s) this PR fixes
Fixes https://github.com/InftyAI/llmaz/issues/65
Special notes for your reviewer
Does this PR introduce a user-facing change?