Change circuit to return Unhealthy

uber-archive / hyperbahn

Service discovery and routing for large scale microservice operations

MIT License

396 stars 57 forks source link

Change circuit to return Unhealthy #240

Closed Raynos closed 8 years ago

Raynos commented 8 years ago

This changes the CircuitBreaker to return an Unhealth error frame instead of a Declined error frame.

This will:

Stop retries on circuit broken; to avoid retry storm
Make graphs a bit nicer because Unhealthy is only used by circuit breaker.

r: @rf @kriskowal @jcorbin

jcorbin commented 8 years ago

Please make it configurable in a way similar to how circuit shorting is configured: on cn~sn~arg1 tuples, with simple wildcarding. In same push, it will serve to shift the boundary away from func (Circuit) shouldRequest() bool to func (Circuit) getRequestError() (code, message string).

rf commented 8 years ago

lgtm

jcorbin commented 8 years ago

Recording our offline convo: please make it actually flippable per circuit, not just for all circuits.

jcorbin commented 8 years ago

This looks :ok: to me now; ideally we'd split out:

simplify the shorts config (originally I had it setup to thread thru, but then backed out to referencing off root only)
change the circuit api to return error frame / reject with it
the rest?

But I can live with this as is if you're pressed for time.