search

PromtEngineer / localGPT

Chat with your documents on your local device using GPT models. No data leaves your device and 100% private.
Apache License 2.0
19.73k stars 2.2k forks source link

Model - Information out of documents #370

Open aliammar182 opened 1 year ago

aliammar182 commented 1 year ago

I have been able to use it and it is producing good results.

But the issue is, this model would also produce answers even if the content is not there in documents, is there a way to handle this?

aabalke33 commented 1 year ago

You will have to alter the prompt template on run_localGPT.py. I use: template = """You are an AI assistant for answering questions about {subject}. Provide a very detailed comprehensive academic answer. If you don't know the answer, just say "I'm not sure." Don't try to make up an answer. If the question is not about {subject} and not directly in the given context, politely inform them that you are tuned to only answer questions about {subject}. Question: {question} ========= {context} ========= Answer:"""

For subject just enter what you are wanting it to answer questions about.

aliammar182 commented 1 year ago

Hey I tried your prompt but it didn't work.

I have also tried making up a lot of templates, none of them seem to work.

Here's the scenario:

I have documents on subject = War and Defense but I have modified documents like replacing "Winston Churchill" with "Redchilli Abraham". So there is no mention of Winston Churchill in the documents at all

So when I ask it about Winston Churchill, it still gives me answers (factually correct) even tho that name is not mentioned in the documents at all.

alexfilothodoros commented 1 year ago

Hi,

Are you recreating the database after replacing the names?

aliammar182 commented 1 year ago

The initial database that I created already had names replaced, you can say my original files are already altered

aabalke33 commented 1 year ago

Is your pipeline temperature set to 0? And are you using the "TheBloke/Llama-2-7B-Chat-GGML" model?

aliammar182 commented 1 year ago

Yes, temperature is set to 0,

I have tried TheBloke/Llama-2-7B-Chat-GGML, it gives responses like "Redchilli is a fictional character so I don't have any information about it"

I also tried "Gpt4all groovy" and it gives somewhat better answers but still has access to its own global knowledge.

renegrob commented 1 year ago

@aliammar182 Can you check if load_single_document returns the modified part of the document? If not, it might be related to #217, meaning that load_single_document should always return all document chunks and not only the first one. Another way to find out would be that it works, if your modification is in the first paragraph of the document.