Chatbot - explore GCP AI model

[ShellyXueHan]

Describe the issue PoC on different AI models with further training with platform info data.

Definition of done

• [x] index doc website
• [x] import digital.gov website with IDIR protected data
• [x] import stackoverflow content
• [x] create scripts for data export
• [ ] upload and update data store with new data
• [x] data clean up with format parsing
• [x] test the chatbot with different models
• [x] dialogFlow tuning
• [ ] test response templating

[ShellyXueHan]

currently blocked on adding additional vector store, waiting for updates from GCP.

[ShellyXueHan]

GCP suggested testing out "Chat" type agent, which accepts multiple datastores. Currently have the following:

• public doc website: directly website indexing
• private website: generated html files with JSONL metadata that contains URLs
• stackoverflow: csv output with questions and verified answers

[ShellyXueHan]

Doesn't seem to be able to use the stackoverflow datastore yet, waiting for feedback from GCP.

Talked to PHSA about their chatbot implementation, they suggested heavy data prepping for better results. Going to ask GDX team for more support!

[ShellyXueHan]

still waiting for feedback from GCP. Brad had an idea to test chunking the document, e.g. remove the header and footer for cleaner content from html. Will try that out!

[ShellyXueHan]

Updates:

• URL metadata is available as part of the response
• with stackoverflow added, the result seems way better. But still need to come up with comprehensive testing set
• going to be creating documentation about GenAI RAG journey on EA's doc website when it's ready

[ShellyXueHan]

Updates:

• currently being blocked with data store updates