error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0')

[MatthewAraujo]

Im using Windows and im getting this error.

this is my repo: https://github.com/MatthewAraujo/chatpdf

I can upload in S3 buckets, and save the pdf in my /tmp folder but when get in the part of vectorizing and embedding individual documents (step 3) I get this type of error. I already saw the video again and copied some code of your repo but didn't work.

`error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37)

TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37)

error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37)

error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3)

error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3)

error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) gs.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 0)`

I appreciated already

[MrFadiAi]

Now i have the same issue after solving the tmp issue

[MrFadiAi]

any news Matthew ? please let me know if u solve it

[Victor-Timi]

Now i have the same issue after solving the tmp issue

How did you solve the tmp issue?

[MatthewAraujo]

Now i have the same issue after solving the tmp issue

How did you solve the tmp issue?

Here https://github.com/Elliott-Chong/chatpdf-yt/issues/2

[AdilCHBALY]

i got the same issue and it indicates that you've exceeded your usage quota for the OpenAI API. This error is related to rate limiting or usage limitations imposed by OpenAI, and it's not a code-level error.

[MatthewAraujo]

So you think we can't do this with we don't pay for Chatgpt 4?

[AdilCHBALY]

(Use node --trace-warnings ... to show where the warning was created) [OPEN AI] Unexpected response structure: { error: { message: 'You exceeded your current quota, please check your plan and billing details.', type: 'insufficient_quota', param: null, code: 'insufficient_quota' } }

After showing the response of createEmbedding it looks like yeah u need to upgrade your OPENAI API Plan i will look more about this and try to find a solution to this matter

[MatthewAraujo]

Thank you my friend

[AdilCHBALY]

ok i found a solution to this issue : You're getting error 429 because either you used all your free tokens or 3 months have passed since you signed up. or you have an account that has a similar phone number to an existing account

Note: If you signed up earlier (e.g., in December 2022), you got $18 worth of free tokens.

Check your API usage in the openAI Dashboard

For example, my free trial expires 4 months and this is what I see right now in the usage dashboard:

so the way to fix this is the create a second account with different phone number in order to get free trial usage again

and your dashboard would look like this

hope it works for you and start coding again ^^

[MatthewAraujo]

ok i found a solution to this issue :

You're getting error 429 because either you used all your free tokens or 3 months have passed since you signed up.

or you have an account that has a similar phone number to an existing account

Note: If you signed up earlier (e.g., in December 2022), you got $18 worth of free tokens.

Check your API usage in the openAI Dashboard

For example, my free trial expires 4 months and this is what I see right now in the usage dashboard:

so the way to fix this is the create a second account with different phone number in order to get free trial usage again

and your dashboard would look like this

hope it works for you and start coding again ^^

Thank you Adil, I will do this right now, I was so interesting in that chatpdf.

Thank so much

[AdilCHBALY]

glad to help

[MrFadiAi]

glad to help

For me is the same issue , but i have already linked my account to my credit card and set limit to the usage, i still have the same error as Matthew

[AdilCHBALY]

glad to help

For me is the same issue , but i have already linked my account to my credit card and set limit to the usage, i still have the same error as Matthew

can you console.log(result) that you get from the req.json() and tell me what you get

[MrFadiAi]

glad to help

For me is the same issue , but i have already linked my account to my credit card and set limit to the usage, i still have the same error as Matthew

can you console.log(result) that you get from the req.json() and tell me what you get

I dont get what do u acutlly mean because i'm a beginner and i'm learning .. but i have all the logs for u here :

Log from Chrome console:

Logs from vscode terminal :

Please migrate your code to use AWS SDK for JavaScript (v3). For more information, check the migration guide at https://a.co/7PzMCcy uploads/1694370324814Converted_Amazon.pdf Converted_Amazon.pdf downloading s3 into file system (node:3048) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3). For more information, check the migration guide at https://a.co/7PzMCcy (Use node --trace-warnings ... to show where the warning was created) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 4) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 4) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 0) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 0) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 6) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 6) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 5) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 5) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 7) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 7) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 8) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 8) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 9) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 9) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 13) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 13) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 12) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 12) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 11) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 11) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 10) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 10) uploads/1694370385440Converted_Amazon.pdf Converted_Amazon.pdf downloading s3 into file system error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 5) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 5) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 4) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 4) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 8) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 8) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 0) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 0) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 9) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 9) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 12) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 12) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 6) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 6) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 10) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 10) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 11) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 11) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 13) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 13) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 7) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:18:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 7) (node:20788) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3). For more information, check the migration guide at https://a.co/7PzMCcy (Use node --trace-warnings ... to show where the warning was created)

[Victor-Timi]

I am also having this issue

[AdilCHBALY]

`import {OpenAIApi,Configuration} from 'openai-edge'

const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY })

const openai = new OpenAIApi(config)

export async function getEmbeddings(text:string){ try { const res = await openai.createEmbedding({ model:'text-embedding-ada-002', input: text.replace(/\n/g," ") }) const result = await res.json() if (result.data && Array.isArray(result.data) && result.data.length > 0 && result.data[0].embedding) { return result.data[0].embedding as number[]; } else { console.log("[OPEN AI] Unexpected response structure:", result); throw new Error("Unexpected response from OpenAI API"); } } catch (error) { console.log("[OPEN AI]",error); throw error } }`

i want you to put this code in embeddings.ts and give me the console log in vscode @fadi9696

[MrFadiAi]

@AdilCHBALY

Thanks for your quick answer

i have did replace it but it seems like i have another issue, it says that my api key in incorrect :

[OPEN AI] Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 26) error embedding document Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 26) [OPEN AI] Unexpected response structure: { error: { message: 'Incorrect API key provided: sk-7MS4DQWLg. You can find your API key at https://platform.openai.com/account/api-keys.', type: 'invalid_request_error', param: null, code: 'invalid_api_key' } } [OPEN AI] Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 27) error embedding document Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 27) [OPEN AI] Unexpected response structure: { error: { message: 'Incorrect API key provided: sk-7MS4DQWLg. You can find your API key at https://platform.openai.com/account/api-keys.', type: 'invalid_request_error', param: null, code: 'invalid_api_key' } } [OPEN AI] Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 29) error embedding document Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 29)

I know for 100% its correct and i have set the usage to $10

ANd here is how my .env looks like

But the thing is .. the error code says Incorrect API key provided: sk-7MS4D**QWLg

And mine is : sk-Ya4Qx**Lq3K

i dont get ti

[AdilCHBALY]

@fadi9696 this issue may be caused by your browser's cache. Sometimes, your browser may hold onto an outdated version of your API key, which can cause this error message to appear. After clearing your browser's cache,can you try your request again and tell me what you get.

[MrFadiAi]

@AdilCHBALY I did removed my browser cache and tried also brave + edge, i get the same error , SO WEIRD

if u need somthing to see, i can send you whatever u want

[AdilCHBALY]

@fadi9696 There might be a chance that AutoGPT is trying to use an old / deleted API key that was saved in your system envitonment if you using a Mac , you have to remove it from your system

[MatthewAraujo]

`import {OpenAIApi,Configuration} from 'openai-edge'

const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY })

const openai = new OpenAIApi(config)

export async function getEmbeddings(text:string){ try { const res = await openai.createEmbedding({ model:'text-embedding-ada-002', input: text.replace(/\n/g," ") }) const result = await res.json() if (result.data && Array.isArray(result.data) && result.data.length > 0 && result.data[0].embedding) { return result.data[0].embedding as number[]; } else { console.log("[OPEN AI] Unexpected response structure:", result); throw new Error("Unexpected response from OpenAI API"); } } catch (error) { console.log("[OPEN AI]",error); throw error } }`

i want you to put this code in embeddings.ts and give me the console log in vscode @fadi9696

I just tested this code and got these results:

[OPEN AI] Unexpected response structure: { error: { message: 'You exceeded your current quota, please check your plan and billing details.', type: 'insufficient_quota', param: null, code: 'insufficient_quota' } }

I didn't make a new Open IA account but that's the replying

[AdilCHBALY]

import {OpenAIApi,Configuration} from 'openai-edge' const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY }) const openai = new OpenAIApi(config) export async function getEmbeddings(text:string){ try { const res = await openai.createEmbedding({ model:'text-embedding-ada-002', input: text.replace(/\n/g," ") }) const result = await res.json() if (result.data && Array.isArray(result.data) && result.data.length > 0 && result.data[0].embedding) { return result.data[0].embedding as number[]; } else { console.log("[OPEN AI] Unexpected response structure:", result); throw new Error("Unexpected response from OpenAI API"); } } catch (error) { console.log("[OPEN AI]",error); throw error } } i want you to put this code in embeddings.ts and give me the console log in vscode @fadi9696

I just tested this code and got these results:

[OPEN AI] Unexpected response structure: { error: { message: 'You exceeded your current quota, please check your plan and billing details.', type: 'insufficient_quota', param: null, code: 'insufficient_quota' } }

I didn't make a new Open IA account but that's the replying

in order to remove this issue you need a new fresh account with different phone number so you get a free trial 5$

[MatthewAraujo]

import {OpenAIApi,Configuration} from 'openai-edge' const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY }) const openai = new OpenAIApi(config) export async function getEmbeddings(text:string){ try { const res = await openai.createEmbedding({ model:'text-embedding-ada-002', input: text.replace(/\n/g," ") }) const result = await res.json() if (result.data && Array.isArray(result.data) && result.data.length > 0 && result.data[0].embedding) { return result.data[0].embedding as number[]; } else { console.log("[OPEN AI] Unexpected response structure:", result); throw new Error("Unexpected response from OpenAI API"); } } catch (error) { console.log("[OPEN AI]",error); throw error } } i want you to put this code in embeddings.ts and give me the console log in vscode @fadi9696

I just tested this code and got these results: [OPEN AI] Unexpected response structure: { error: { message: 'You exceeded your current quota, please check your plan and billing details.', type: 'insufficient_quota', param: null, code: 'insufficient_quota' } } I didn't make a new Open IA account but that's the replying

in order to remove this issue you need a new fresh account with different phone number so you get a free trial 5$

Here, I just got a new account and tested

the results:

`Module not found: Can't resolve 'encoding' in 'C:\Users\matth\Área de Trabalho\projetos\IAs\chatpdf\node_modules\node-fetch\lib'

Import trace for requested module: ./node_modules/node-fetch/lib/index.js ./node_modules/cross-fetch/dist/node-ponyfill.js ./node_modules/cross-fetch/dist/node-polyfill.js ./node_modules/@pinecone-database/pinecone/dist/v0/index.js
./node_modules/@pinecone-database/pinecone/dist/index.js
./src/lib/pinecone.ts ./src/app/api/create-chat/route.ts

• warn ./node_modules/node-fetch/lib/index.js Module not found: Can't resolve 'encoding' in 'C:\Users\matth\Área de Trabalho\projetos\IAs\chatpdf\node_modules\node-fetch\lib'

Import trace for requested module: ./node_modules/node-fetch/lib/index.js ./node_modules/cross-fetch/dist/node-ponyfill.js ./node_modules/cross-fetch/dist/node-polyfill.js ./node_modules/@pinecone-database/pinecone/dist/v0/index.js
./node_modules/@pinecone-database/pinecone/dist/index.js
./src/lib/pinecone.ts ./src/app/api/create-chat/route.ts

• warn ./node_modules/node-fetch/lib/index.js Module not found: Can't resolve 'encoding' in 'C:\Users\matth\Área de Trabalho\projetos\IAs\chatpdf\node_modules\node-fetch\lib'

Import trace for requested module: ./node_modules/node-fetch/lib/index.js ./node_modules/cross-fetch/dist/node-ponyfill.js ./node_modules/cross-fetch/dist/node-polyfill.js ./node_modules/@pinecone-database/pinecone/dist/v0/index.js
./node_modules/@pinecone-database/pinecone/dist/index.js
./src/lib/pinecone.ts ./src/app/api/create-chat/route.ts (node:8660) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy (node:8660) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
For more information, check the migration guide at https://a.co/7PzMCcy uploads/1694386095970Currículoo-Vitae.pdf Currículo Vitae.pdf downloading s3 into file system (node:16456) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3).
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:59:28) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:59:28) at async Promise.all (index 3) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:46:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error embedding document Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:59:28) at async Promise.all (index 3) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:46:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37)

• wait compiling...
• event compiled client and server successfully in 1393 ms (2973 modules) (node:8660) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Please migrate your code to use AWS SDK for JavaScript (v3). For more information, check the migration guide at https://a.co/7PzMCcy `

[AdilCHBALY]

@MatthewAraujo thats good then your OPENAI is working i got the same issues as you have now with pinecode Error upserting chunk [PineconeError: PineconeClient: Error calling upsert: PineconeError: The requested feature 'Namespaces' is not supported by the current index type 'Starter'.]

[MatthewAraujo]

I changed my package.json library for the ones that he is doing on the video and got new errors:

[OPEN AI] Unexpected response structure: { error: { message: 'Rate limit reached for default-text-embedding-ada-002 in organization org-Q1YOsy3oqDjPbV2N874zGx4v on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.', type: 'requests', param: null, code: 'rate_limit_exceeded' } }

I think I pass this type of error but right now it just works with the API plus limite

[AdilCHBALY]

hahaha wow this is a new error i will look into it

[MatthewAraujo]

hahaha wow this is a new error i will look into it

I think we solved the other issue but right now just works if you increase the rate limit, I'm a little sad about it because I'm from Brazil so a dollar for us here is 5x REAL R$ so I will not able to get the plus plain, but the projects its really interesting

[Victor-Timi]

`import {OpenAIApi,Configuration} from 'openai-edge'

const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY })

const openai = new OpenAIApi(config)

export async function getEmbeddings(text:string){ try { const res = await openai.createEmbedding({ model:'text-embedding-ada-002', input: text.replace(/\n/g," ") }) const result = await res.json() if (result.data && Array.isArray(result.data) && result.data.length > 0 && result.data[0].embedding) { return result.data[0].embedding as number[]; } else { console.log("[OPEN AI] Unexpected response structure:", result); throw new Error("Unexpected response from OpenAI API"); } } catch (error) { console.log("[OPEN AI]",error); throw error } }`

i want you to put this code in embeddings.ts and give me the console log in vscode @fadi9696

Thanks,

I just tried your code, but now I am getting this:

[OPEN AI] Unexpected response structure: { error: { message: 'Rate limit reached for default-text-embedding-ada-002 in organization org-MHYrLrMNqc25gf1eKqpzPNKe on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.', type: 'requests', param: null, code: 'rate_limit_exceeded' } } [OPEN AI] Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 240) error embedding document Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:22:19)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 240)

[Victor-Timi]

Here is the solution

Just add this: const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, });

[MrFadiAi]

const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, });

Hey Victor, could u tell me how did u fix it ?

i tried your code an i get red line

[AdilCHBALY]

@fadi9696 make sure you import it from import { RecursiveCharacterTextSplitter } from '@pinecone-database/doc-splitter'

[MrFadiAi]

Hey @AdilCHBALY , the code from yesterday that you have send, it seems like it glitching or somthing, i get another api code that not belong to me and it does not recognize it, So i put the orginal code back and added the RecursiveCharacterTextSplitter

here is my code :

import { OpenAIApi, Configuration } from "openai-edge"; import { RecursiveCharacterTextSplitter } from '@pinecone-database/doc-splitter' const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY, });

const openai = new OpenAIApi(config); const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, }); export async function getEmbeddings(text: string) { try { const response = await openai.createEmbedding({ model: "text-embedding-ada-002", input: text.replace(/\n/g, " "), }); const result = await response.json(); return result.data[0].embedding as number[]; } catch (error) { console.log("error calling openai embeddings api", error); throw error; } }

here is the error :

error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9)
at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9)
at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21)
at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9)
at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 5) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 5) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 3) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 3) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 0) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 0) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 9) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 9) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 8) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 8) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 6) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 6) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 12) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 12) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 4) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 4) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 7) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 7) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 11) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 11) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 1) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 1) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 13) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 13) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 10) error embedding document TypeError: Cannot read properties of undefined (reading '0')
at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28)
at async Promise.all (index 10) (node:9844) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

[Victor-Timi]

Hey @AdilCHBALY , the code from yesterday that you have send, it seems like it glitching or somthing, i get another api code that not belong to me and it does not recognize it, So i put the orginal code back and added the RecursiveCharacterTextSplitter

here is my code :

import { OpenAIApi, Configuration } from "openai-edge"; import { RecursiveCharacterTextSplitter } from '@pinecone-database/doc-splitter' const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY, });

const openai = new OpenAIApi(config); const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, }); export async function getEmbeddings(text: string) { try { const response = await openai.createEmbedding({ model: "text-embedding-ada-002", input: text.replace(/\n/g, " "), }); const result = await response.json(); return result.data[0].embedding as number[]; } catch (error) { console.log("error calling openai embeddings api", error); throw error; } }

here is the error :

error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 2) at async loadS3IntoPinecone (webpack-internal:///(rsc)/./src/lib/pinecone.ts:45:21) at async POST (webpack-internal:///(rsc)/./src/app/api/create-chat/route.ts:31:9) at async eval (webpack-internal:///(rsc)/./node_modules/next/dist/server/future/route-modules/app-route/module.js:254:37) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 5) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 5) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 3) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 0) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 0) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 9) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 9) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 8) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 8) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 6) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 6) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 12) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 12) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 4) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 4) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 7) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 7) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 11) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 11) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 1) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 13) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 13) error calling openai embeddings api TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 10) error embedding document TypeError: Cannot read properties of undefined (reading '0') at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embeddings.ts:24:27) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:56:28) at async Promise.all (index 10) (node:9844) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.

Try uploading another file, and make it small file, it's going to work.

I think the token limit for GPT3:5 is a lot small, except you upgrade to 4 that's when you can be able to upload larger file

[MrFadiAi]

hey @Victor-Timi , i tried with smaller pdf , same problem..

did you have run it? is it working with you ?

[apurvjha123]

can anyone see this why this error ??

@AdilCHBALY @Victor-Timi @Elliott-Chong

[Victor-Timi]

can anyone see this why this error ??

@AdilCHBALY @Victor-Timi @Elliott-Chong

Check your Pinecone API connection, import PineconeClient and not Pinecone.

import {PineconeClient} from “@pinecone-database/pinecone”

[apurvjha123]

No it's Pinecone

On Tue, 19 Sep 2023, 23:57 Victor Timi, @.***> wrote:

can anyone see this why this error ??

[image: Screenshot 2023-09-19 185618] https://user-images.githubusercontent.com/100297235/268972205-1bdec32e-c74a-4da3-92a0-fd7f0ebdba47.png

@AdilCHBALY https://github.com/AdilCHBALY @Victor-Timi https://github.com/Victor-Timi @Elliott-Chong https://github.com/Elliott-Chong

Check your Pinecone API connection, import PineconeClient and not Pinecone.

import {PineconeClient} from @.***/pinecone”

— Reply to this email directly, view it on GitHub https://github.com/Elliott-Chong/chatpdf-yt/issues/3#issuecomment-1726271811, or unsubscribe https://github.com/notifications/unsubscribe-auth/AX5GUEZU5RU6UVIVTS2DBF3X3HPZXANCNFSM6AAAAAA4RN75AU . You are receiving this because you commented.Message ID: @.***>

[Victor-Timi]

okay so, have you fixed it?

[yash-prajapati19]

Here is the solution

Just add this: const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, });

I Tried after adding this but still getting this error :

[OPEN AI] Unexpected response structure: { error: { message: 'Rate limit reached for default-text-embedding-ada-002 in organization org-7Td8ozoUVekXqY3tKYM9cnwP on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.', type: 'requests', param: null, code: 'rate_limit_exceeded' } } [OPEN AI] Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embedding.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:53:28) at async Promise.all (index 5) error embedding document Error: Unexpected response from OpenAI API at getEmbeddings (webpack-internal:///(rsc)/./src/lib/embedding.ts:22:19) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async embedDocument (webpack-internal:///(rsc)/./src/lib/pinecone.ts:53:28) at async Promise.all (index 5)

Here Is my code:

Pinecone.ts ->

import { Pinecone, PineconeRecord } from "@pinecone-database/pinecone"; import { downloadFromS3 } from "./s3-server"; import { PDFLoader } from "langchain/document_loaders/fs/pdf"; import md5 from "md5"; import { Document, RecursiveCharacterTextSplitter, } from "@pinecone-database/doc-splitter"; import { getEmbeddings } from "./embedding"; import { convertToAscii } from "./utils";

export const getPineconeClient = () => { return new Pinecone({ environment: process.env.PINECONE_ENVIRONMENT!, apiKey: process.env.PINECONE_API_KEY!, }); };

type PDFPage = { pageContent: string; metadata: { loc: { pageNumber: number }; }; };

export async function loadS3IntoPinecone(fileKey: string) { // 1. obtain the pdf -> downlaod and read from pdf console.log("downloading s3 into file system"); const file_name = await downloadFromS3(fileKey); if (!file_name) { throw new Error("could not download from s3"); } console.log("loading pdf into memory" + file_name); const loader = new PDFLoader(file_name); const pages = (await loader.load()) as PDFPage[];

// 2. split and segment the pdf const documents = await Promise.all(pages.map(prepareDocument)); console.log(documents);

// 3. vectorise and embed individual documents const vectors = await Promise.all(documents.flat().map(embedDocument));

// 4. upload to pinecone const client = await getPineconeClient(); const pineconeIndex = await client.index("pdf-ai"); const namespace = pineconeIndex.namespace(convertToAscii(fileKey));

console.log("inserting vectors into pinecone"); await namespace.upsert(vectors);

return documents[0]; }

async function embedDocument(doc: Document) { try { const embeddings = await getEmbeddings(doc.pageContent); console.log("embeddingd:", embeddings); const hash = md5(doc.pageContent);

console.log({
  id: hash,
  values: embeddings,
  metadata: {
    text: doc.metadata.text,
    pageNumber: doc.metadata.pageNumber,
  },
} as PineconeRecord);

return {
  id: hash,
  values: embeddings,
  metadata: {
    text: doc.metadata.text,
    pageNumber: doc.metadata.pageNumber,
  },
} as PineconeRecord;

} catch (error) { console.log("error embedding document", error); throw error; } }

export const truncateStringByBytes = (str: string, bytes: number) => { const enc = new TextEncoder(); return new TextDecoder("utf-8").decode(enc.encode(str).slice(0, bytes)); };

async function prepareDocument(page: PDFPage) { let { pageContent, metadata } = page; pageContent = pageContent.replace(/\n/g, ""); // split the docs const splitter = new RecursiveCharacterTextSplitter({ chunkSize: 1000, chunkOverlap: 200, }); const docs = await splitter.splitDocuments([ new Document({ pageContent, metadata: { pageNumber: metadata.loc.pageNumber, text: truncateStringByBytes(pageContent, 36000), }, }), ]); return docs; }

Here is my embedding.ts code :

import { OpenAIApi, Configuration } from "openai-edge";

const config = new Configuration({ apiKey: process.env.OPENAI_API_KEY, });

const openai = new OpenAIApi(config);

export async function getEmbeddings(text: string) { try { const res = await openai.createEmbedding({ model: "text-embedding-ada-002", input: text.replace(/\n/g, " "), }); const result = await res.json(); if ( result.data && Array.isArray(result.data) && result.data.length > 0 && result.data[0].embedding ) { return result.data[0].embedding as number[]; } else { console.log("[OPEN AI] Unexpected response structure:", result); throw new Error("Unexpected response from OpenAI API"); } } catch (error) { console.log("[OPEN AI]", error); throw error; } }

And Yes it is working with 1-2 page pdfs (around 100kb), But not working with large files like more then 4 pages pdfs

@Victor-Timi , @AdilCHBALY