search

redpanda-data / benthos

MIT License
213 stars 61 forks source link

Support circuit breaking inputs #77

Open AndreasBergmeier6176 opened 2 months ago

AndreasBergmeier6176 commented 2 months ago

When processing from a FIFO input stream a fail in a connected REST Service should possibly trigger a circuit breaking of the input stream. Might be necessary for Inputs to declare internally, whether they support Circuit Breaking natively though. Config could be:

processors:
- circuit_breaker:
    half_open_max_requests: 2
    interval: "1s"
    open_timeout: 60s
    trip: |
      root = this.requests >= 3 && (this.total_failures / this.requests) >= 0.6
    processors:
    - http:
mihaitodor commented 2 months ago

Thanks for the enhancement request @AndreasBergmeier6176! There was a similar proposal in the past: https://github.com/redpanda-data/connect/issues/1069 Unfortunately, that didn't make it as a feature. What does half_open_max_requests represent and what is a trip? Would you happen to have some examples from other systems where this is implemented?

AndreasBergmeier6176 commented 2 months ago

Thanks for the enhancement request @AndreasBergmeier6176! There was a similar proposal in the past: redpanda-data/connect#1069 Unfortunately, that didn't make it as a feature. What does half_open_max_requests represent and what is a trip? Would you happen to have some examples from other systems where this is implemented?

I modeled this mostly after https://github.com/sony/gobreaker

Another quite popular implementation is Envoy: https://www.envoyproxy.io/docs/envoy/latest/configuration/upstream/cluster_manager/cluster_circuit_breakers