Closed BadrLaajali closed 1 month ago
Astriel
commented
2 months ago I'm also having some problems with pgvector. It used to work before, but now, each time I'm trying to make an upsert, I'm receveing a OK response, but nothing into the database while it used to work.
BadrLaajali
commented
2 months ago It's working fine with Pinecone and Qdrant. Maybe it's linked to Postgres pgvector database...
Astriel
commented
2 months ago I've tried with Pinecone, same problem. Flowise says data are upsert, but still nothing in the vector database.
Astriel
commented
2 months ago I'm having the same error if I change my embedding to Bedrock from OpenAI.
jackborst
commented
2 months ago I'm experiencing a similar problem, possibly related to the same root cause.
When using the PostgreSQL Record Manager (RM) and PostgreSQL Vector Store, changing the "cleanup" parameter in the RM module from "none" to "full" results in a "Not implemented" error.
Steps to reproduce:
Error message: Error: Error: Error: Not implemented.
Additional observations:
msiwanHahn
commented
2 months ago Hi, same issue as described above (https://github.com/FlowiseAI/Flowise/issues/3004#issue-2462195755 & https://github.com/FlowiseAI/Flowise/issues/3004#issuecomment-2296254124).
Flowise hosted in Azure. Managed PG in Azure, too.
Flow: upsert API Chatflow.json
Additionally the Record Manager keeps PG connections in IDLE "ClientRead" for an unlimited amount of time.
We are reading PDFs from a file share and using the upsert API, same occurs when using the Flowise UI and Upload File from the PDF Loader Node.
HenryHengZJ
commented
1 month ago Here's what I have tried:
OpenAI Embeddings + Postgres Vector Store + Postgres Record Manager
My local Postgres is using Docker, and Flowise is using local git installation
# Compose a postgres database together with the extension pgvector
services:
db:
hostname: 127.0.0.1
image: pgvector/pgvector:pg15
ports:
- 5432:5432
restart: always
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=1234
volumes:
- ./init.sql:/docker-entrypoint-initdb.d/init.sqlStart the postgres docker using docker compose up
Then
1.) Have a document store with following upsert config:
2.) Upsert data successfully:
3.) Change to Full:
4.) Works correctly as it is able to skipped upserting same documents:
5.) Change to Incremental
6.) See "Not Implemented" error:
So I was able to replicate the error only in Incremental, looks like its opposite from what you observed @jackborst
HenryHengZJ
commented
1 month ago Update: this PR https://github.com/FlowiseAI/Flowise/pull/3180 fixes this
Describe the bug When attempting to upsert data from a Notion database into a Postgres vector database using pgvector, an error occurs during the upsert process after modifying a page in Notion. The error message states "Error: vectorsService.upsertVector - Error: Error: Not implemented." Despite this error, new duplicate records are created in the vector database, indicating that the system is not properly recognizing or updating existing records. This occurs even though a Postgres Record Manager is being used to detect modifications in the flow.
To Reproduce Steps to reproduce the behavior:
Expected behavior The Postgres Record Manager should detect the modifications in the Notion pages. The upsert process should then recognize existing records based on this information and update them instead of creating duplicates.
Screenshots
Setup
Additional context The issue persists even when switching the Record Manager to Supabase. The error occurs specifically during the upsert process after modifying existing Notion pages. The log shows successful initialization of components (Recursive Character Text Splitter, Cohere Embeddings, Notion Database, Postgres Record Manager) before encountering the error. The error seems to be originating from the Postgres_VectorStores.upsert method, suggesting a potential issue with the implementation of the upsert functionality in the Postgres vector store component. Log details :