search

pashpashpash / vault-ai

OP Vault ChatGPT: Give ChatGPT long-term memory using the OP Stack (OpenAI + Pinecone Vector Database). Upload your own custom knowledge base files (PDF, txt, epub, etc) using a simple React frontend.
https://vault.pash.city
MIT License
3.25k stars 306 forks source link

How accurate is this? #1

Open Uday-461 opened 1 year ago

Uday-461 commented 1 year ago

How accurate will this be? Will it just do a search on the data and return the closest possible match? Or can I upload a large amount of data and have it analyse the data to give me inference from one possible document from the data?

Uday-461 commented 1 year ago

Also is it compatible with GPT3.5 turbo, or GPT4, or both?

DoingFedTime commented 1 year ago

Your API key from OpenAI is probably for 3.5 Turbo, so...

gitardav commented 1 year ago

I compared response results. See issue here: https://github.com/pashpashpash/vault-ai/issues/26#issue-1676657944

isConic commented 1 year ago

By default it only grabs 3 of what it thinks are the most related documents and tries to stuff contents into a single query.

Do not treat this as a summarizer. Do not assume that it takes into account/thinks-about or processes ALL of your documents.
Of potentially thousands of uploaded documents. It will grab 3 and stuff as much of the content from those three documents into the query as it can while fitting the 4097 token allowance. In some cases it will cut some of those documents out if they're too lengthy.

isConic commented 1 year ago

Worth Noting. It will grab 3 documents based on wether or not the documents, when summarized via embeddings, are semantically similar to your query.