yvann-ba
commented
1 year ago Hi, Indeed I spent many hours on this problem, in fact ConversationalretrievalChain is quite unstable, because since the last maj langchain has changed the arguments it takes, the arguments that allowed to change easily the prompt template have been replaced by condense_prompt_template and qa_prompt which are really not practical. I think I'll try to switch to a custom agent to have more control over the prompt template, I should normally be able to associate memory+retriever+custom prompt with initialize_agent, I could then restrict the agent to the file (another chain might do the trick but agents seem to have more future than chains in terms of scalability). Don't hesitate if you have ideas to contribute, I would be happy to collaborate. Have a nice day !
philashworth
Right now, even if I ask questions unrelated (off topic questions), I will get a reply on that question since it's from chatgpt.
Expectations are to get a reply saying that it's not related to the topic.
I've done some testing to add the lines below to the QA_PROMPT (used by mayooear/gpt4-pdf-chatbot-langchain): "If you don't know the answer, just say you don't know. DO NOT try to make up an answer. If the question is not related to the context, politely respond that you are tuned to only answer questions that are related to the context. "
However, answer either changes drastically OR it no longer able to understand that it's taking context from a file.
I'm not sure whether I'm doing it right, or maybe is there any way to change how semantic search is being done?
Lastly, thank you so much for sharing this project. If there's any direction that I could look into, I will help experiment and contribute/share my findings.