SupabaseVectorStore: custom fields are omitted in addDocuments

[tom-bywild]

Description

addDocuments in SupabaseVectorStore uses its addVectors to store data in Supabase, but addVectors removes all custom fields from the document. Only content, embeddings, metadata are kept.

In a somewhat more complex project, the DB-schema might include other fields as well (therefore extending the Document-interface from Langchain.js). Using the current implementation with Langchain.js doesn't allow storing those custom fields. A workaround would be to update the entities after addVectors has succeeded, which is not ideal (and could introduce consistency issues within the application).

Potential solution

A fix would be to simply not filterout all non-Document related fields in addVectors:

- const rows = vectors.map((embedding, idx) => ({
-   content: documents[idx].pageContent,
-   embedding,
-   metadata: documents[idx].metadata,
- }));

+ const rows = vectors.map((embedding, idx) => {
+   const { pageContent, ...rest } = documents[idx];
+   return {
+     embedding,
+     content: pageContent,
+     ...rest,
+    };
+ });

[dosubot[bot]]

Hi, @tom-bywild,

I'm helping the langchainjs team manage their backlog and am marking this issue as stale. It looks like you opened this issue regarding the addDocuments method in SupabaseVectorStore removing custom fields from the document when using addVectors. The proposed solution involves modifying the addVectors method to retain all non-Document related fields when storing data. There haven't been any further developments or comments on the issue.

Could you please confirm if this issue is still relevant to the latest version of the langchainjs repository? If it is, please let the langchainjs team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[sajas-nm]

why is this issure closed ? i am still facing this issue