search

hugapi / hug

Embrace the APIs of the future. Hug aims to make developing APIs as simple as possible, but no simpler.
MIT License
6.86k stars 389 forks source link

circuit breakers support #146

Open timothycrosley opened 8 years ago

timothycrosley commented 8 years ago

Hug should ideally have built-in support for the circuit breakers pattern as it is a very core component of any modern resilient micro-service.

https://en.wikipedia.org/wiki/Circuit_breaker_design_pattern

ryananguiano commented 8 years ago

I am very interested in this as well. I have looked at this issue before, and it just seems really hard to implement correctly in Python. A good circuit breaker would keep track of all requests across all threads/processes/greenlets and keep a global state.

Netflix has made Hystrix (https://github.com/Netflix/Hystrix/wiki/How-it-Works) and this wiki seems to explain how they solved some of the harder problems. The circuit breaker would almost certainly need to run in its own thread and manage worker thread pools as well. But since Hystrix is written for the JVM, I feel a Python implementation would run into a lot of performance issues from threading.

If you want to maybe help settle on a road map for this, I wouldn't mind taking a stab at the overall structure.

timothycrosley commented 8 years ago

@ryananguiano, as an alternative approach, I think you could get most of the benefits simply by taking advantage of a cache store that is available to all thread / processes (such as Redis / Memecached). I think conservatively in 99.9999% of the time, you really don't care if a race condition causes two requests to fail at the same time, and then cause the circuit breaker to take effect, so a non-locking pattern could be used for very good performance. An option could be added to the circuit breaker to enable locks for those that really need it.

What do you think about this approach?

ryananguiano commented 8 years ago

Hmm... I think requiring an external store for this feature would make it a lot harder to implement in production, but I think it should be pluggable if you would like to choose one of those stores. The other thing about using an external store, it has to keep a lot of stateful data and this in itself will generate a lot of overhead. And if that store happens to be across the network, then it misses the whole point.

Even if it suffers from performance, I think a pure Python implementation would fit the best into the Falcon ecosystem. I think multiprocessing will be the way to do it. If the system does not support it, we could of course always use multiprocessing.dummy which is just a wrapper around threading. But then again, this might be overkill and async/await might be able to handle it just fine.

timothycrosley commented 8 years ago

@ryananguiano, 100% agree that the data-store should be pluggable if used. I think the big problem I see with a pure Python implementation is that when using a standard like UWSGI you don't really have 100% control over how processes / threads are separated, these settings are defined by the actual server (a separate entity) and the end-users configuration.

timothycrosley commented 8 years ago

Also, as a counter point: circuit breakers are defined to handle all sorts of issues, I think the rarest of those is loss of network connectivity to everything (including all internal services) it's probably is perfectly reasonable to use a central, over internal network, cache and then define special logic when even that is down.

ryananguiano commented 8 years ago

I did find this which is pretty simple - https://github.com/danielfm/pybreaker/blob/master/src/pybreaker.py

I would still add a way to specify a cache and a fallback per each circuit breaker. Also, asynchronous responses are a huge requirement for me.

Can you think of any other features that should be present for a first draft?

timothycrosley commented 8 years ago

@ryananguiano I think that would be more then sufficient for an initial draft, and provide a tremendous amount of value. BTW thanks for adding your input and researching this issue it's very appreciated.

ryananguiano commented 8 years ago

So I was looking at a couple ways to do this, and I see valid use cases for different people needing either multiprocessing, threading, or concurrent tasks. So I think the best way to handle this is to make an interface that passes the necessary process info to whichever engine the user selects in the settings. The default should just be async concurrency so it only uses one process unless someone specifically configures it otherwise.

I have a couple questions to see which direction you want to go.

I'm going to be pulling a lot of influence from Hystrix and will make the most basic unit of work a CircuitCommand.

We can also have generic commands that will cover most use cases like a CircuitURLRequest or CircuitSocketRequest. We would then implement these in hug/use.py and also allow people to use in their code.

horvathcom commented 8 years ago

Have you considered pybreaker? I don't know anything about it. I was just curious about the topic. On Feb 26, 2016 3:07 AM, "Ryan Anguiano" notifications@github.com wrote:

So I was looking at a couple ways to do this, and I see valid use cases for different people needing either multiprocessing, threading, or concurrent tasks. So I think the best way to handle this is to make an interface that passes the necessary process info to whichever engine the user selects in the settings. The default should just be async concurrency so it only uses one process unless someone specifically configures it otherwise.

I have a couple questions to see which direction you want to go.

-

I think this feature alone is going to add a lot of weight to the package. Do you think we should make it a subpackage that is an optional install? We could make it a separate package like hug-circuitbreaker and you can add it to your setup.py as an alternate install so it can be installed like hug[circuitbreaker].

How do you feel about using external packages like pebble https://pypi.python.org/pypi/Pebble or gevent http://www.gevent.org? We would use these packages to create the different "engines" that each connect our interface.

I'm going to be pulling a lot of influence from Hystrix https://github.com/Netflix/Hystrix and will make the most basic unit of work a CircuitCommand.

We can also have generic commands that will cover most use cases like a CircuitURLRequest or CircuitSocketRequest. We would then implement these in hug/use.py and also allow people to use in their code.

— Reply to this email directly or view it on GitHub https://github.com/timothycrosley/hug/issues/146#issuecomment-189175260.

ryananguiano commented 8 years ago

Ya. I looked at that already. It seems to be a good start but seems to be missing some functionality that I require. I've been looking at this problem in Python for quite a while and I feel that I can architect a solid foundation that will be scalable in production.