Stevenic / vectra

Vectra is a local vector database for Node.js with features similar to pinecone but built using local files.
MIT License
321 stars 29 forks source link

LocalDocumentIndex: Add option to avoid saving text of local docs #20

Open Stevenic opened 9 months ago

Stevenic commented 9 months ago

The LocalDocumentIndex currently saves a copy of the text of any document added to the index to its index folder. That makes sense for text fetched from the web but not as much for local documents. There maybe times when you want to save a snapshot of the local document to the index but the default should be to reference local documents in-place...

GaureeshAnvekar commented 6 months ago

For this, I wanted to confirm if it's the following line that needs to be replaced. If yes, do we just save the entire path to the original file in the index folder?

inside the file "LocalDocumentIndex.ts" line #232 -> vectra/src/LocalDocumentIndex.ts

await fs.writeFile(path.join(this.folderPath,${documentId}.txt), text);