search

pashpashpash / vault-ai

OP Vault ChatGPT: Give ChatGPT long-term memory using the OP Stack (OpenAI + Pinecone Vector Database). Upload your own custom knowledge base files (PDF, txt, epub, etc) using a simple React frontend.
https://vault.pash.city
MIT License
3.26k stars 307 forks source link

Document Size Issue #2

Closed DuckyDough closed 1 year ago

DuckyDough commented 1 year ago

I'm having an issue persisting my embeddings to my vector DB. Whenever I upload a pdf or text file I hit a quote error Reason: Error getting embeddings: error, status code: 429, message: You exceeded your current quota, please check your plan and billing details.

I tried a 61MB pdf and then a 500kb txt file and both yielded the same error message. What size documents do you recommend? Do you prepare your data in any way before upload? Or is a paid plan for Pinecone needed to persist book-sized ( or larger) text files?

pashpashpash commented 1 year ago

Are you running it locally?

DuckyDough commented 1 year ago

Yep I am, provided my own api keys and managed to get the UI and server running.

DuckyDough commented 1 year ago

I added a payment method to my openai account and this seems to have resolved the quota issue. However, now when I try the same files I hit a 2023/04/18 12:46:55 [UploadHandler ERR] Error upserting embeddings to Pinecone: Post "quickstart-*******.us-east4-gcp.pinecone.io/vectors/upsert": unsupported protocol scheme "" I can raise a separate issue if you'd prefer

DuckyDough commented 1 year ago

The unsupported protocol scheme issue was solved by prepending https:// to the pinecone endpoint address. The app is working correctly. Thank you

slagernate commented 1 year ago

@pashpashpash a mention to prepend https:// in the README would be nice