GaiaNet-AI / gaianet-node

Install and run your own AI agent service
https://www.gaianet.ai/
GNU General Public License v3.0
2.4k stars 70 forks source link

feat: support empty snapshot #47

Closed apepkuss closed 1 month ago

apepkuss commented 1 month ago

Major changes:

juntao commented 1 month ago

I have two issues when the snapshot field is empty:

  1. When we start llama-api-server.wasm, we still need to start both the chat and embedding models. The /embedding endpoint should work in this setup. Currently, only the chat model is started.
  2. The gaianet init command fails to update the gaianet-domain/frpc.toml file with the node ID from nodeid.json or config.json.
apepkuss commented 1 month ago

@juntao

  1. Different from rag-api-server, llama-api-server only supports a single model, chat or embedding, instead of both at the same time.
  2. This PR only introduces a minor change in init to check if snapshot is empty or not before calling create_collection function. No code change covers the registry.
juntao commented 1 month ago
  1. Different from rag-api-server, llama-api-server only supports a single model, chat or embedding, instead of both at the same time.

I think we need to get to the point where llama-api-server can support a chat and an embedding model. Using the chat model to serve the /embedding API endpoint is meaningless.

apepkuss commented 1 month ago

Ok. We need to provide support for the following three scenarios in llama-api-server, right?

  1. Only chat model
  2. Only embedding model
  3. Both chat and embedding models
juntao commented 1 month ago

Ok. We need to provide support for the following three scenarios in llama-api-server, right?

  1. Only chat model
  2. Only embedding model
  3. Both chat and embedding models

Yes. But for a MVP, 1 + 3 would suffice.

apepkuss commented 1 month ago

Ok. To satisfy the requirements, we have to refactor the CLI and the handlers in llama-api-server and release a new version. I'll close this PR first and reopen it once llama-api-server gets ready.