[BUG] Socket closed unexpectedly - Redis

[danielstankw]

I have installed the Flowiseai using the traditional method (docker) as well as Kubernetes pods using unofficial chart. In both of those cases whenever I attempt to connect to the redis store to store vector embeddings I get the following error:

2024-04-15 07:03:35 [INFO]: Starting Flowise...
2024-04-15 07:03:35 [INFO]: ⚡️ [server]: Flowise Server is listening at 3000
2024-04-15 07:03:36 [INFO]: 📦 [server]: Data Source is being initialized!
2024-04-15 07:03:38 [INFO]: 📦 [server]: Data Source has been initialized!
2024-04-15 07:24:32 [INFO]: ⬆️ POST /api/v1/vector/internal-upsert/962166d0-816d-4c8d-8a2e-341a53495b37
2024-04-15 07:25:32 [ERROR]: uncaughtException:  Socket closed unexpectedly
Error: Socket closed unexpectedly
    at Socket.<anonymous> (/usr/local/lib/node_modules/flowise/node_modules/@redis/client/dist/lib/client/socket.js:194:118)
    at Object.onceWrapper (node:events:632:26)
    at Socket.emit (node:events:517:28)
    at TCP.<anonymous> (node:net:350:12)

Not sure if this is the issue but it seems like its Redis communication problem: https://stackoverflow.com/questions/71574823/reusing-redis-connection-socket-closed-unexpectedly-node-redis https://community.fly.io/t/redis-socketclosedunexpectedlyerror-socket-closed-unexpectedly/5897

[danielstankw]

Do I need to install any additional dependencies to run flowiseai with redis?

I am accessing Redis that is hosted externally. When I use In memory vector store I am also failing to upsert with different error (see below)

2024-04-16 13:21:32 [ERROR]: Error: Connection error.
Error: Error: Connection error.
    at InMemoryVectorStore_VectorStores.upsert (/usr/local/lib/node_modules/flowise/node_modules/flowise-components/dist/nodes/vectorstores/InMemory/InMemoryVectorStore.js:26:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:342:37)
    at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:117:32)
    at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:6:9)
    at async createInternalUpsert (/usr/local/lib/node_modules/flowise/dist/controllers/vectors/index.js:26:16)
2024-04-16 13:21:32 [ERROR]: [server]: Error: Error: Error: Connection error.
Error: Error: Error: Connection error.
    at buildFlow (/usr/local/lib/node_modules/flowise/dist/utils/index.js:405:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async upsertVector (/usr/local/lib/node_modules/flowise/dist/utils/upsertVector.js:117:32)
    at async Object.upsertVectorMiddleware (/usr/local/lib/node_modules/flowise/dist/services/vectors/index.js:6:9)
    at async createInternalUpsert (/usr/local/lib/node_modules/flowise/dist/controllers/vectors/index.js:26:16)

I tried: Enter container and ping the AzureOpenAi and Redis endpoints, I also tried to run test scripts for retrieving data from redis and embeddings from within the container and was successful.

Do you have any suggestion of pipeline that will let me test if the issue is with embedding or vector store?

[HenryHengZJ]

Strange, you should be fine with InMemory since it doesn't make connection. Id suggest try with different vector store, then embeddings, basically divide and conquer

[danielstankw]

@HenryHengZJ The Issue persists and I boiled it down to the problem with redis vector store - flowiseai implementation.
I checked here, and I believe it might be due to the timeout on the redis connection, but I am not 100% sure.

I think fix requires modification of redis module implementation in flowiseai, swapping vector store to the In-memory solved the connection problem.

Even if I dont execute any queries there error below happens.

2024-05-23 08:10:08 [ERROR]: uncaughtException:  Socket closed unexpectedly
Error: Socket closed unexpectedly
    at Socket.<anonymous> (/usr/local/lib/node_modules/flowise/node_modules/@redis/client/dist/lib/client/socket.js:194:118)
    at Object.onceWrapper (node:events:632:26)
    at Socket.emit (node:events:517:28)
    at TCP.<anonymous> (node:net:350:12)

[danielstankw]

@HenryHengZJ is there any update on the redis vector store fix?