Hi! I was testing how pybreaker would behave in various scenarios where state is being stored on Redis and the Redis server misbehaves.
CircuitRedisStorage will handle intermittent and complete connection loss with the Redis server in an acceptable manner. But it doesn't handle data loss on the Redis server gracefully. When no value has been set for a key, the Redis library returns None. This results in an AttributeError being thrown on this line:
https://github.com/danielfm/pybreaker/blob/c5cb8fa93749657d6d59342b7145aaee2607b28d/src/pybreaker.py#L476
I opened a PR (#25) which aims to resolve this by doing the following when it finds missing data on Redis:
Resetting the state on Redis to the fallback circuit state
Reset the fail counter
Also, perhaps the circuit breaker's listeners should be notified with a custom exception when this occurs? I'm open to suggestions.
danielfm
commented
6 years ago
Hi! I was testing how pybreaker would behave in various scenarios where state is being stored on Redis and the Redis server misbehaves.
CircuitRedisStorage will handle intermittent and complete connection loss with the Redis server in an acceptable manner. But it doesn't handle data loss on the Redis server gracefully. When no value has been set for a key, the Redis library returns None. This results in an AttributeError being thrown on this line: https://github.com/danielfm/pybreaker/blob/c5cb8fa93749657d6d59342b7145aaee2607b28d/src/pybreaker.py#L476
I opened a PR (#25) which aims to resolve this by doing the following when it finds missing data on Redis:
Also, perhaps the circuit breaker's listeners should be notified with a custom exception when this occurs? I'm open to suggestions.