search

Stevenic / vectra

Vectra is a local vector database for Node.js with features similar to pinecone but built using local files.
MIT License
321 stars 29 forks source link

Unexpected non-whitespace character after JSON at position XYZ #43

Open KrishnaManohar1997 opened 5 months ago

KrishnaManohar1997 commented 5 months ago

Hey @Stevenic , Facing the issue of SyntaxError when trying to store huge chunks of data (async) After storing certain data points, it starts to break.

Stack Trace

SyntaxError: Unexpected non-whitespace character after JSON at position 10395 (line 1 column 10396)
    at JSON.parse (<anonymous>)
    at LocalIndex.<anonymous> (/Users/krishnamanohar/ManoharCode/PromptIntExtension/node_modules/vectra/lib/LocalIndex.js:345:31)
    at Generator.next (<anonymous>)
    at fulfilled (/Users/krishnamanohar/ManoharCode/PromptIntExtension/node_modules/vectra/lib/LocalIndex.js:28:58) {stack: 'SyntaxError: Unexpected non-whitespace charac…/node_modules/vectra/lib/LocalIndex.js:28:58)', message: 'Unexpected non-whitespace character after JSON at position 10395 (line 1 column 10396)'}

IndexJson Please observe the end of the JSON index file that's getting malformed

image

Do let me know if I need to share any other details for us to debug this issue.

Stevenic commented 4 months ago

So if you’re not doing so already you should call beginUpdate() before doing large scale adds as your updates will be faster and that may address the issue you’re seeing. Call endUpdate() after your last add to commit the changes to disk.

I’m going to be offline for the next 2 weeks but please let me know if that resolves your issue.

KrishnaManohar1997 commented 4 months ago

Thanks, I will give this a try and will update the result here.