search

Yiiipu / SimplePDF

Simple MacOS PDF reader with ChatGPT integrated. You can chat with your local PDF. The returned response can cite the page number to ensure correctness.
GNU General Public License v3.0
81 stars 1 forks source link

q: 16k context q: conversation history implementation #1

Open brezl8 opened 1 year ago

brezl8 commented 1 year ago

hello! two quick questions: 1) how exactly did you implement the conversation history? 2) how can i use the new gpt3.5-turbo-16k model instead of the standard gpt3.5-turbo or gpt4 model?

thank you so much for your time!

Yiiipu commented 1 year ago

Hi,

  1. I implement it in the swift side, which has not been open sourced yet. In short, it will transform the question you ask by adding the necessary details from history into it.
  2. You can directly edit the api.py in this repo. Simply change the model's name.
brezl8 commented 1 year ago

thanks a lot. one more quick question; in the pdfGPT repo it is mentioned that it uses a "Deep Averaging Network Encoder" as a superior solution to openAI embeddings. Is this up to date? are the embeddings better than the ones used by openAI?

thank you!

Yiiipu commented 1 year ago

Hi, This is a very good question. I will try OpenAI embeddings to see how it would work. I believe the current one is not superior but there should not be a big difference in performance in this task.

brezl8 commented 1 year ago

hi! i am thinking forth and back what tool i should implement for my use case and i am repeatedly coming back to your repository / implementation. could you briefly talk a bit more about the conversation history implementation? just how exactly the relevant history is determined and added to the question throughout the conversation? and what's your experience how token efficient this is exactly? i'm more interested in the process and less so in the exact code, so that i can understand whether this would work well for my use case or not. thank you so much!

Yiiipu commented 1 year ago

Hi, I add another layer of ChatBot who receives several latest messages (pruned) and convert user's newest question to a complete one. For example, if your first question is "who is Jennifer Lewis?", and your second question is "what is her research direction?", normally pdfGPT will be confused by the second question because it does not know who is her. So, the first layer of ChatBot I add can take the chat history and transform the second question to "what is Jennifer Lewis's research direction?" and send to pdfGPT. It still costs token to perform this task but costs less than other ways.

I am not sure if I explain this well. We could discuss more or collaborate if you need.