2.2.1 Setup prompts and conduct Output Testing with knowledge base

[earth2travis]

Conduct comprehensive testing of the outputs generated by the pipeline and Gaia node. The testing process will include validating the accuracy and completeness of the data embeddings, ensuring that the system performs as expected under different conditions. Identifying and addressing any issues during this phase is crucial for delivering a reliable and robust solution.

• [x] Connect embeddings snapshot to gaia node
• [x] Setup prompt
• [x] Perform queries through the chat ui
• [ ] Identify potential improvements
• [ ] Try fine-tuning an LLM model

[santteegt]

Gaia config params:

• prompt_template
  • prompt_type var on start script
  • available templates link
• system_prompt
• rag_prompt
• rag_policy
  • either system-message or last-user-message
  • more info in this link
• reverse_prompt
• qdrant_limit
• qdrant_score_threshold
• embedding_collection_name
• (chat_batch_size, embedding_batch_size)
• (chat_ctx_size, embedding_ctx_size)

Script to start the rag-api-server with snapshot:

cmd=(wasmedge --dir .:./dashboard \
        --env NODE_VERSION=$installer_version \
        --nn-preload default:GGML:AUTO:$chat_model_name \
        --nn-preload embedding:GGML:AUTO:$embedding_model_name \
        rag-api-server.wasm \
        --model-name $chat_model_stem,$embedding_model_stem \
        --ctx-size $chat_ctx_size,$embedding_ctx_size \
        --batch-size $chat_batch_size,$embedding_batch_size \
        --prompt-template $prompt_type,embedding \
        --rag-policy $rag_policy \
        --qdrant-collection-name $embedding_collection_name \
        --qdrant-limit $qdrant_limit \
        --qdrant-score-threshold $qdrant_score_threshold \
        --web-ui ./ \
        --socket-addr 0.0.0.0:$llamaedge_port)
        --rag-prompt" "$rag_prompt" # if not empty
        --reverse-prompt "$reverse_prompt" # if not empty

rag-api-server entrypoint:

• source code