redis storage open state initialization?

[yu-le]

Hi danielfm,

Does the redis storage is intended as the shared breaker state storage for different workers? it seems like the current implementation would be stayed in open state. For example: start 32 workers, each worker with state as closed, if one worker enter state open, it will store the state in the redis. when another worker receive the request, it will compare its state name with redis storage, and enter open state, but in the CircuitOpenState:

 def __init__(self, cb, prev_state=None, notify=False):
        """
        Moves the given circuit breaker `cb` to the "open" state.
        """
        super(CircuitOpenState, self).__init__(cb, STATE_OPEN)
        self._breaker._state_storage.opened_at = datetime.utcnow()
        if notify:
            for listener in self._breaker.listeners:
                listener.state_change(self._breaker, prev_state, self)

It did not read redis storage opened_at, but set it as current time, it cause the closed worker has no way to enter half-open state?

[phillbaker]

I think this is related to https://github.com/danielfm/pybreaker/issues/15.

What we ended up doing is similar to:

BREAKER = CircuitBreaker(...)
...
    def active(self):
        timeout = timedelta(seconds=BREAKER.reset_timeout)
        timeout_active = not hasattr(BREAKER.state, 'opened_at') or \
            hasattr(BREAKER.state, 'opened_at') and datetime.now() < BREAKER.state.opened_at + timeout
        return BREAKER.current_state == 'closed' or not timeout_active

[danielfm]

Closed by #27, feel free to re-open this if you have problems.

[vt-iwamoto]

I started using this patch. It seems to work fine for now.

import pybreaker

def circuit_breaker_open(self):
    with self._lock:
        self.state = self._state_storage.state = pybreaker.STATE_OPEN
        self._state_storage.opened_at = pybreaker.datetime.utcnow()

def circuit_open_state_init(self, cb, prev_state=None, notify=False):
    super(pybreaker.CircuitOpenState, self).__init__(cb, pybreaker.STATE_OPEN)
    try:
        _ = self._breaker._state_storage.opened_at
    except:
        self._breaker._state_storage.opened_at = pybreaker.datetime.utcnow()
    if notify:
        for listener in self._breaker.listeners:
            listener.state_change(self._breaker, prev_state, self)

pybreaker.CircuitBreaker.open = circuit_breaker_open
pybreaker.CircuitOpenState.__init__ = circuit_open_state_init