search

sensu / sensu-transport

The Sensu transport abstraction library.
MIT License
14 stars 19 forks source link

Redis transport duplicate checks #31

Open sstarcher opened 8 years ago

sstarcher commented 8 years ago

Servers are running checks five times. I have dropped the redis cache and restart all of the sensu clients and server.

Sensu 0.24.1

Jun 17 15:12:50 ip-10-0-20-137.ec2.internal docker[3901]: {"timestamp":"2016-06-17T15:12:50.460799+0000","level":"warn","message":"previous check command execution in progress","check":{"name":"consul_leader","issued":1466176370,"command":"check.rb"}}
Jun 17 15:12:50 ip-10-0-20-137.ec2.internal docker[3901]: {"timestamp":"2016-06-17T15:12:50.460994+0000","level":"warn","message":"previous check command execution in progress","check":{"name":"consul_leader","issued":1466176370,"command":"check.rb"}}
Jun 17 15:12:50 ip-10-0-20-137.ec2.internal docker[3901]: {"timestamp":"2016-06-17T15:12:50.461083+0000","level":"warn","message":"previous check command execution in progress","check":{"name":"consul_leader","issued":1466176370,"command":"check.rb"}}
Jun 17 15:12:50 ip-10-0-20-137.ec2.internal docker[3901]: {"timestamp":"2016-06-17T15:12:50.461229+0000","level":"warn","message":"previous check command execution in progress","check":{"name":"consul_leader","issued":1466176370,"command":"check.rb"}}
portertech commented 7 years ago

@sstarcher what version of Redis are you using? What does your Redis deployment look like?

sstarcher commented 7 years ago

@portertech This is an AWS elasticache redis instance

Engine: 2.8.24

I'm using the AWS defaults for 2.8. I can get a list of them if you want.

portertech commented 7 years ago

We need to investigate TCP connection error/failure vs Redis error and the difference in handling.

portertech commented 7 years ago

Sensu 0.27 will have several sensu-transport Redis improvements and fixes, the release is expected to drop in the next 2 weeks. We are still able to reproduce duplicate check results when triggering an Elasticache HA failover :-1:

shajithravi commented 5 years ago

@portertech Any updates on using redis as a transport for Sensu core 1.5/1.6 ?