gptscript-ai / desktop

MIT License
22 stars 13 forks source link

Knowledge - Not able to get information from knowledge file when specifying the name of document to use. #320

Closed sangee2004 closed 1 month ago

sangee2004 commented 2 months ago

Electron build from commit id - a8d8c5c39b3

Steps to reproduce the problem

  1. Start a chat with Tildy
  2. Add the following knowledge file using "Add Knowledge" - MyFeb12.pdf
  3. Ask this question - " What can I eat in empty stomach"
  4. Knowledge tool will not get used for this and we get the response from LLM directly.
  5. Ask to use the file - "Use MyFeb12 pdf file for this"
  6. This results in call being made to Knowledge tool , but the response presented are not from the document even though LLM claims it is from the this document. Screenshot 2024-08-27 at 5 30 15 PM

    MyFeb12.pdf

Screenshot 2024-08-27 at 5 29 55 PM
iwilltry42 commented 2 months ago

The LLM decided to use the filename as a keyword parameter.. which is kind of bad, as supposedly the content doesn't contain the filename :confounded: As we now have true hybrid search, I'm going to remove that tool.gpt parameter altogether :+1:

iwilltry42 commented 2 months ago

Large changes to how knowledge is integrated: #385 -> this removed the param.

Anyway, asking to specifically check a file by name is not expected to do anything right now - it won't make the results better.

sangee2004 commented 1 month ago

Tested with build - a66cd29e I donot see know_retrieve_keywords parameter being used forknowledgeRetrieval tool call.

I am able to get information from knowledge file without having to specify the knowledge file name.

Even if I specify the name, i dont see it getting used by the knowledgeRetrieval tool which is the expected behavior now.

Screenshot 2024-09-09 at 11 40 20 AM Screenshot 2024-09-09 at 11 40 05 AM