Some files not being processed to generate embeddings

[jpchauny]

Hi,

Some files are not being processed to generate embeddings. What could be the reasons? How can I check this? I was observing these files and could not find a pattern to find out why they are not being processed.

[brianpetro]

Hi @jpchauny

Thanks for trying Smart Connections.

You can open the console log (CTRL+SHIFT+I on Windows) to see if any errors are occuring.

Additionally, do you have billing setup on your OpenAI account? If not, rate limits on free accounts often cause this.

Let me know if that helps, Brian 🌴

[jpchauny]

I have a GPT Plus Account, paying 20 USD/month.

This is one error (of several similar ones) from the console:

undefined plugin:smart-connections:1123 Error: Request failed, status 400 at new t (app.js:1:1668026) at tU (app.js:1:1668218) at app.js:1:1668895 at app.js:1:235927 at Object.next (app.js:1:236032) at a (app.js:1:234750) plugin:smart-connections:1026 failed embedding batch plugin:smart-connections:1116 undefined plugin:smart-connections:1123 Error: Request failed, status 400 at new t (app.js:1:1668026) at tU (app.js:1:1668218) at app.js:1:1668895 at app.js:1:235927 at Object.next (app.js:1:236032) at a (app.js:1:234750)

[jpchauny]

I just configured a paid account and still getting the same errors.

[brianpetro]

Sometimes the rate limits take a bit of time to be lifted after setting up the paid account. And the ChatGPT Plus is a separate billing system so that time starts when you setup the paid account.

Try it out in an hour or so and let me know if that helps.

Thanks for following up!

Brian 🌴

[jpchauny]

Getting the same errors after 5 hours of having set up the paid account.

[brianpetro]

Same error codes?

[MarkSant]

I'm facing the same error in the same context/conditions.... is there any information you need to help you understand what's going on?

[MarkSant]

Updating my last message. After some trials, I've fount out that two things were causing the failure of embeddings: 1 - in YAML, when setting aliases, I was using 2 aliases for thoses notes, but this was beeing done by setting "alias: "........" , "........"". In this case, the " " was needed because it was a tittle and, therefore, had comma and dots in it. The fact is whe problem was solved when I setted the same configuration as: aliases:

• .........
• ..... For some reason the files with embbedding problems were solved and they got embbeded after all.

2 - I've noticed that "-" or ":" would too cause the error, so I took those caracters out.

In the end, just one note is still getting the error and all the problem was centered at the caracters and strutucture of YAML frontmatter used in my notes.

Hope this helps solving the question.

[brianpetro]

@MarkSant thanks for following up 🌴

[brianpetro]

Let me know if this is still an issue in v2, closing for now 😊🌴

[brianpetro]

Recorded response https://youtu.be/Cr1jEftMuZM