Knowledge - actually inject context based on user input

gptscript-ai / desktop

MIT License

22 stars 13 forks source link

Knowledge - actually inject context based on user input #326

Open cjellick opened 3 months ago

cjellick commented 3 months ago

From a convo with Thorsten:

Right now, it is pretty naive: we have a static context that says something dumb like "you have an ad hoc rag tool..." and then we rely on the LLM deciding to call retreival

I think it's time to do a proper "Retrieval Augmented Generation" wherein we write a context tool that looks at the input from the user, figures out if knowledge has relevant information about that input, and if it does, go ahead and add it as context.

My success criteria is:

That the quality for when there IS a match stays the same as it is now - so to test it ill do question similar to ones ive done in the past and expect results to be as good or better
That the behavior when there IS NOT a match is better - right now i feel you have to coax the assistant to answer from its training data or from the internet tool if the knowledge tool is present
I dont have to coax it into using calling konwledge. its completely transparent

iwilltry42 commented 3 months ago

Ref https://github.com/gptscript-ai/knowledge/issues/93

iwilltry42 commented 3 months ago

Keeping some notes here:

https://github.com/gptscript-ai/gptscript/blob/1c75954594d59abe8ae371bc7c96ce953eec4101/pkg/runner/runner.go#L347 is the point where the GPTScript input (user chat input) gets lost when getting the context, as it's not JSON-formatted.
The above means, that even at this point, where the GPTSCRIPT_INPUT env var should be added, the input is already empty.

iwilltry42 commented 2 months ago

Large changes to how knowledge is integrated: #385

sangee2004 commented 2 months ago

Tested with build from b6c00b5330eb

We now see Knowledge Retrieval Context being added when there are queries made relating to knowledge files.

While testing this feature , there were couple of instances were I see the queries made relating to the knowledge file are not being answered from the knowledge file. In these cases , output from knowledgeInstruction is empyt ( No data retrieved from knowledge base.).

Stack Trace shows this:

Detailed steps to reproduce the issue is tracked in the following 2 issues: https://github.com/gptscript-ai/desktop/issues/336#issuecomment-2338898813 https://github.com/gptscript-ai/desktop/issues/402#issuecomment-2339237256