Closed apepkuss closed 1 month ago
I have two issues when the snapshot field is empty:
llama-api-server.wasm
, we still need to start both the chat and embedding models. The /embedding
endpoint should work in this setup. Currently, only the chat model is started.gaianet init
command fails to update the gaianet-domain/frpc.toml
file with the node ID from nodeid.json
or config.json
. @juntao
rag-api-server
, llama-api-server
only supports a single model, chat or embedding, instead of both at the same time.init
to check if snapshot
is empty or not before calling create_collection
function. No code change covers the registry.
- Different from
rag-api-server
,llama-api-server
only supports a single model, chat or embedding, instead of both at the same time.
I think we need to get to the point where llama-api-server
can support a chat and an embedding model. Using the chat model to serve the /embedding
API endpoint is meaningless.
Ok. We need to provide support for the following three scenarios in llama-api-server
, right?
Ok. We need to provide support for the following three scenarios in
llama-api-server
, right?
- Only chat model
- Only embedding model
- Both chat and embedding models
Yes. But for a MVP, 1 + 3 would suffice.
Ok. To satisfy the requirements, we have to refactor the CLI and the handlers in llama-api-server
and release a new version. I'll close this PR first and reopen it once llama-api-server
gets ready.
Major changes:
llama-api-server.wasm
into installer and CLI tool