FlowiseAI / Flowise

Drag & drop UI to build your customized LLM flow
https://flowiseai.com
Apache License 2.0
30.29k stars 15.65k forks source link

[BUG] vectorsService.upsertVector Not implemented (Postgres Record Manager) #2200

Closed enriquesb closed 1 month ago

enriquesb commented 5 months ago

When attempting to upsert a new document using a Postgres Record Manager, I encountered the following error: "Error: vectorsService.upsertVector - Error: Error: Not implemented."

To Reproduce Steps to reproduce the behavior:

  1. Connect a Postgres Record Manager element with the "Full" option selected for the "Clean up" parameter, to a Postgres vector store.
  2. Upsert the content of a .docx file using Recursive Character text splitter.
  3. Change/upload another file in the Docx File loader.
  4. Save the chatflow and attempt to upsert the new document.

Expected behavior I expected the content of the first document to be erased from the database, followed by the upsertion of the new document's content into the database.

Screenshots image

Flow postgres record manager upsert error Chatflow.json

HenryHengZJ commented 5 months ago

is it just postgres record manager?

enriquesb commented 5 months ago

I only use postgres databases. So I can't speak about the other record managers.

17lxve commented 4 months ago

is it just postgres record manager?

I've had the same issue with Postgres, In-memory vector store, and Pinecone. It seems like the embeddings just fail when i load more than a few documents in.

jimmy8646 commented 4 months ago

I also encountered the same issue. I tried to clear the database, and encountered the same error message when performing an Upsert.

jconey87 commented 4 months ago

Im getting upsert failed on everything i try with vector db's. image

kyuumeitai commented 4 months ago

Im getting upsert failed on everything i try with vector db's. image

Hi there, I think that your error is something else: your error says that "fetch failed". Is ollama up and running?

kyuumeitai commented 4 months ago

I have this error as well. It happens to me if I use Postgres Record Manager but also if I use MySQL Record Manager.

The error logs are this:

2024-05-14 20:32:45 [ERROR]: Error: Not implemented.
Error: Error: Not implemented.
    at Postgres_VectorStores.upsert (/usr/local/lib/node_modules/flowise/node_modules/flowise-components/dist/nodes/vectorstores/Postgres/Postgres.js:82:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:389:37)
    at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:117:32)
    at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:9:16)
    at async createInternalUpsert (/usr/local/lib/node_modules/flowise/dist/controllers/vectors/index.js:28:29)
2024-05-14 20:32:45 [ERROR]: [server]: Error: Error: Error: Not implemented.
Error: Error: Error: Not implemented.
    at buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:453:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:117:32)
    at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:9:16)
    at async createInternalUpsert (/usr/local/lib/node_modules/flowise/dist/controllers/vectors/index.js:28:29)

It's the same if I use Mysql or Postgres Record Manager. The only way that I can use it is to set the cleanup to "none"

EDIT: Can confirm that is related to Postgres retriever. I have implemented a Chroma retriever and the MySQL and Postgres Record Manager works OK.

oiqm4T commented 3 months ago

Error still occurring when using PG record manager with PG vector. Error: Error: Not implemented. at Postgres_VectorStores.upsert

sergiozaninotti commented 2 months ago

same error here :(

enriquesb commented 2 months ago

I managed to get my chatflow working by following a YouTube video by Leon van Zyl. I connected the record manager block to Supabase, which resolved the issue. It seems that the error I encountered earlier was due to incorrect configuration of my local PostgreSQL database.

I apologize for any confusion caused.

saadaziz0014 commented 1 month ago

I followed the same instruction from leon van zyl video number 6 but still got same error with in memory, chromadb and pinecode vector store

jofranjp commented 3 weeks ago

My setup was postgres as Vector Store and Record Manager, and I got the error mentioned by the author of this thread. This only happened if the Cleanup option is either set to incremental or Full. I was not able to fix this.

But when I change my Vector Store to Qdrant and still the Record Manager is Postgres, everything working fine. The cleanup setting of the Record Manager is still set Incremental.

HenryHengZJ commented 3 weeks ago

This PR should fixes this: https://github.com/FlowiseAI/Flowise/pull/3180 using Postgres Record Manager + Postgres Vector Store

jofranjp commented 3 weeks ago

Great job Henry. How can we get the update? Is that part of the Flowise release update?

orsharon7 commented 4 days ago

I am experiencing the same issue:

Error: vectorsService.upsertVector - Error: Error: No vectors provided

I tried removing the connection and restarting the service, however it didnt solve it. it happens when I try to upsert another PDF file..

Here is my architecture:

Screenshot 2024-09-29 at 11 19 17