Facing Issue with RAG

[codemaster1104]

I was running Llama-3-8b-Instruct as gaianet node with my self created snapshot which I successfully created without any issues. This Snapshot is created by chunking book Rust Programing Language the chunked txt file used for creating snapshot can be seen here

Also while gianet init and gaianet start commands I didnt face any issues

`akshat@akshat:~/gaianet$ gaianet init [+] Checking the config.json file ...

[+] Downloading Meta-Llama-3-8B-Instruct-Q5_K_M.gguf ...

• Using the cached Meta-Llama-3-8B-Instruct-Q5_K_M.gguf in /home/akshat/gaianet

[+] Downloading nomic-embed-text-v1.5.f16.gguf ...

• Using the cached nomic-embed-text-v1.5.f16.gguf in /home/akshat/gaianet

[+] Creating 'default' collection in the Qdrant instance ...

• Start a Qdrant instance ...

• Remove the existed 'default' Qdrant collection ...

• Download Qdrant collection snapshot ... ######################################################################### 100.0%######################################################################### 100.0% The snapshot is downloaded in /home/akshat/gaianet

• Import the Qdrant collection snapshot ... The process may take a few minutes. Please wait ... Recovery is done!

[+] Preparing the dashboard ... You already have a private key.

[+] Preparing the GaiaNet domain ...

[+] COMPLETED! GaiaNet node is initialized successfully.

To start the GaiaNet node, run the command: gaianet start <<<

akshat@akshat:~/gaianet$ gaianet start [+] Checking the config.json file ...

You already have a private key. [+] Starting Qdrant instance ...

Qdrant instance started with pid: 6684

[+] Starting LlamaEdge API Server ...

Run the following command to start the LlamaEdge API Server:

wasmedge --dir .:./dashboard --env NODE_VERSION=0.1.3 --nn-preload default:GGML:AUTO:Meta-Llama-3-8B-Instruct-Q5_K_M.gguf --nn-preload embedding:GGML:AUTO:nomic-embed-text-v1.5.f16.gguf rag-api-server.wasm --model-name Code_explanation_LLAMA-3-8B-Instruct-GGUF,Nomic-embed-text-v1.5 --ctx-size 4096,512 --batch-size 4096,512 --prompt-template llama-3-chat,embedding --rag-policy system-message --qdrant-collection-name default --qdrant-limit 1 --qdrant-score-threshold 0.5 --web-ui ./ --socket-addr 0.0.0.0:8080 --rag-prompt "You are an expert of the Rust language.\n----------------\n"

LlamaEdge API Server started with pid: 7146

Verify the LlamaEdge API Server. Please wait seconds ...

[+] Starting gaianet-domain ...

gaianet-domain started with pid: 7164

The GaiaNet node is started at: https://0x25b3d984d5c18b672383cc4f6827af11468fa58d.us.gaianet.network

To stop the GaiaNet node, run the command: gaianet stop <<<

akshat@akshat:~/gaianet$ `

Although I am constantly facing this issue that my node is working for general queries but for specific queries(whenever I use Rust in my query) my node uses computing resources but doesn't produce any response

[juntao]

Please look into the ~/gaianet/log/start-llamaedge.log file to see where it stopped.

It seems that when you ask a Rust related question, the RAG returns results from the vector database. Maybe something went wrong in database retrieval.

[codemaster1104]

From my understanding Retrial seems to be working fine:

Following is the start-llamaedge.log

Only ques asked for this case was "What is Rust?"

Doc containing content of log file

[juntao]

As far as I can see, the server completed the RAG and then generated a response. Perhaps it is an issue with JS in your browser? Can you try the server using an API request?

[juntao]

Just use curl to send the API request like this

curl -X POST http://localhost:8080/v1/chat/completions \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system", "content": "You are a helpful assistant."}, {"role":"user", "content": "What is Rust?"}]}'

[codemaster1104]

Yes, it seems to be working in CLI API requests