chameleonbr / node-red-contrib-redis

Node RED client for Redis with pub/sub, list, lua scripting and other commands support.
MIT License
46 stars 40 forks source link

ReplyError: READONLY You can't write against a read only replica. #37

Closed dm7pwnage closed 4 years ago

dm7pwnage commented 4 years ago

Hi,

maybe I may ask for help again:

Best regards

Summary

I have a 5 node redis/sentinel setup.

OK: Setting/getting keys using redis-cli works on master node works properly.

Not OK: Setting a key using node-red connecting to the master node gets error "ReplyError: READONLY You can't write against a read only replica." (This error on the replicas is ok, but not handled in my test flow).

Question

  1. Where is my configuration error?
  2. Would you be able to provide a hint?

Setup

Replication status (current master redis-010301.localdomain).

redis-010301.localdomain:42004> info replication
# Replication
role:master
connected_slaves:4
slave0:ip=192.168.4.180,port=42004,state=online,offset=59750463,lag=0
slave1:ip=192.168.4.185,port=42004,state=online,offset=59750320,lag=0
slave2:ip=192.168.2.180,port=42004,state=online,offset=59750463,lag=0
slave3:ip=192.168.1.180,port=42004,state=online,offset=59750320,lag=2
master_replid:5be7e740c6f0b41d6216fea61cd2b5459e3e6054
master_replid2:71cee693dfa920f146fc15d36836ad11d8031cf9
master_repl_offset:59750606
second_repl_offset:593598
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:58702031
repl_backlog_histlen:1048576
redis-010301.localdomain:42004>

where

redis-010101.localdomain = 192.168.1.180 redis-010201.localdomain = 192.168.2.180 redis-010301.localdomain = 192.168.3.180 redis-010401.localdomain = 192.168.4.180 redis-010501.localdomain = 192.168.4.185

Node configurations are (example for only node 3):

{
    "host": "redis-010301.localdomain",
    "port": 42004,
    "password": "mypass"
}

My test flow

My simple test flow is attached:

image

My simple test flow for import:

redis-flow.zip

chameleonbr commented 4 years ago

Hi, to help you I need to replicate the same config and same redis topology, but I don't have expertise to help what you need. I found on IORedis documentation something related to slave connections(https://github.com/luin/ioredis#sentinel). You know that only master is writable, so on the slaves is imposible use the set command or any command that write on redis server.

dm7pwnage commented 4 years ago

Hello,

thank you for your quick response. I think you do not need to replicate the config at the moment. I simplified it as follows. Maybe this helps.

image

best regards

dm7pwnage commented 4 years ago

I progressed and have additional info.

It works if I have only one Redis servers. If I have > 1 Redis servers, it seems connection is made randomly between configured servers. After several minutes it somehow changes between the configured servers.

It seems all commands are only sent to one server, not to the configured ones.

dm7pwnage commented 4 years ago

I can confirm now.

It is NOT possible to have >1 Redis server per flow.

I had to split Redis handling into 1 flow per Redis server and link logic to individual Redis server handling flows using link in, link out nodes.

chameleonbr commented 4 years ago

Try mark "Block Commands" on redis cmd, and see if now works.

chameleonbr commented 4 years ago

Hi @dm7pwnage, I change the way that connection is created. Now I use the server name except if the block option is checked. Please test version 1.3.5 and verify if now the multiple connections work ok.

dm7pwnage commented 4 years ago

Thank you for your effort and the update. I will schedule tests next weeks. I'll keep you updated.

chameleonbr commented 4 years ago

I'm awaiting your response to close this issue.