Add handlers to deal with HTTP-level details

[justinabrahms]

Right now, the behavior interface deals with sockets, which puts a lot of additional complexity onto the implementor when only wanting to deal with HTTP level things.

My use case is: I have an app that uses local storage as a read-through cache where we return the stale data initially, then refresh it with an async RPC.

I'm having difficulty reproducing caching errors because my local development server is too fast and the refreshing RPC returns at the same time as the localStorage results return. I need to slow down queries to specific endpoints in my app. Ideally, my code would look something like:

import gevent
from vaurien.behaviors import Behavior
from vaurien.behaviors.delay import Delay

class SlowQueryBehavior(Delay):
    """Slows down queries to the backend denoted by being ending in `.json`."""
    name = 'slow_query'
    options = {
        "to_match": ("File ending to match", str, ".json")
    }
    options.update(Delay.options)

    def on_before_http_handle(self, source, backend):
        ext = self.option('to_match')
        if source.endpoint[-len(ext):] == ext:
            gevent.sleep(self.option('sleep'))
        return True

Behavior.register(SlowQueryBehavior)

if __name__ == '__main__':
    # run from command line.
    import sys
    from vaurien.run import main
    main()

As it works now, I'll need to handle reading out of a socket, parsing the HTTP bits to make my decision, then forward it along fixing the drained socket.

[tarekziade]

Yeah that's a good point. What we could add is 2 new functions in the behavior base class:

• on_before_handle: called before the protocol starts its work
• on_before_backend_call : called before the protocol sends the data to the backend
• on_after_backend_call: called before the protocol sends the data to the frontent
• on_after_handle: called before the protocol sends the data to the frontent

on_before_backend_call and on_after_backend_call would be the place where the protocol has transformed incoming and outgoing data into protocol specific data. In the case of HTTP: an http query and response.

So in the case of HTTP, you'd write:

class SlowQueryBehavior(Delay):
    name = 'slow_query'
    options = {
        "to_match": ("File ending to match", str, ".json")
    }
    options.update(Delay.options)

    def on_before_backend_call(self, protocol, source, backend):
        request = protocol.get_data()
        ext = self.option('to_match')

        if request.PATH.endswith(ext):
            gevent.sleep(self.option('sleep'))
        return True

Of course it implies that each protocol documents what .get_data() sends back. How does that sounds ?

@ametaireau what do you think?

[justinabrahms]

This would be ideal for my case. :smile: