search

SonicWarrior1 / pdfchat

Local PDF Chat Application with Mistral 7B LLM, Langchain, Ollama, and Streamlit
MIT License
119 stars 40 forks source link

Unable to get it working #2

Closed shersoni610 closed 10 months ago

shersoni610 commented 11 months ago

Hello,

I ran the code but somehow it never answers anything from the PDF file. It gives answer randomly and without any context. Is there something that I need to do?

SonicWarrior1 commented 11 months ago

Hello,

I ran the code but somehow it never answers anything from the PDF file. It gives answer randomly and without any context. Is there something that I need to do?

The model answers the questions according to the document provided as well as the existing knowledge on which it was trained so it may not give accurate answers everytime.

First try on different pdf files if it behaves same for every file or not.

Also You can try adjusting the prompt according to your need or you can also try playing with the chunk_size and chunk_overlap values in the ReccursiveTextSplitter depending upon the size of document you are trying to provide.

Otherwise if it is still not working just provide me with the pdf file and the questions you are trying to ask and i will try it on my machine.

shersoni610 commented 11 months ago

[Ci_GFPose_Learning_3D_Human_Pose_Prior_With_Gradient_Fields_CVPR_2023_paper.pdf.pdf](https://github.com/SonicWarrior1/pdfchat/files/13540016/Ci_GFPose_Learning_3D_Human_Pose_Prior_With_Gradient_Fields_CVPR_2023_paper.pdf.pdf)

shersoni610 commented 11 months ago

I tried several PDF files and I asked very simple questions such as:

  1. What is the document about?
  2. What is the title?
  3. Can you briefly give a summary?
  4. How many citations this paper uses?
  5. How many images and tables this paper has?

All answers were wrong so I am not sure if this is problem with Mistral or something else.

shersoni610 commented 11 months ago

Do we need to change the temperature value in the mistral to get more reliable answers? If yes, do you know how can we do it?

shersoni610 commented 11 months ago

It looks to me Mistral is far behind Chat GPT-4 for a pdf questions answering. GPT-4 did amazing work.

SonicWarrior1 commented 11 months ago

It looks to me Mistral is far behind Chat GPT-4 for a pdf questions answering. GPT-4 did amazing work.

Offcourse GPT-4 will be better because it got a team of excellent engineers behind and training and optimizing the model just for pdf questions answering. I wouldn't say that Mistral is behind it is actually better performing as compared to other models like Lamma13b but the problem we are getting all depends on what type technique we are using to get the data from PDF documents.

So we are using a concept called Retrival Augmented Generation (RAG) which works as it first converts the document into tokens which can be understandable by the model and storing it into a vector store. The model than fetches the tokens according to questions context and then answers it.

I tried with your pdf file and it is sometimes giving the right answers and sometimes not. So i wouldnt blame the model instead i would blame the technique we are using. I am just using simple RAG implementation in this project and there are alot of complex techniques out there for RAG .

We can try using different techniques and experimenting with different parameters. Also Ollama supports different models other than mistral so you can also try using them.

Machine Learning is sometimes just game of trial and error , just keep experimenting with different things and u might get end up with a good solution for your problem.

https://medium.com/towards-data-science/the-research-agent-4ef8e6f1b741 You can try reading the above article it follows a different and better approach then what i did.