Add ability to simulate faults

[adamdougal]

Heya! Thanks for the library!

Would it be possible to add fault simulation to this library? I've used WireMock in the past in Java projects and it's ability to simulate faults has been really useful https://wiremock.org/docs/simulating-faults/.

I'm happy to contribute where possible if you're happy for this to be included in the library.

Thanks

[adamdougal]

Though, on looking at the code I've seen this is possible, but I have to create my own custom handler:

    def handler(_) -> werkzeug.Response:
        time.sleep(10)
        return werkzeug.Response({"foo": "bar"}, status=200)

    httpserver.expect_request("/foobar").respond_with_handler(handler)

It would be nice if we were able to do something like:

httpserver.expect_request("/foobar").respond_with_json({"foo": "bar"}, delay=1000)
httpserver.expect_request("/foobar").respond_with_json({"foo": "bar"}, fault=Fault.MALFORMED_RESPONSE_CHUNK)

[csernazs]

hi @adamdougal ,

Thanks for the suggestion! I need to check the link you suggested in details, but it sounds ok to add some functinality from there to pytest-httpserver.

Generally speaking, the respond handler callback is for any case which is not covered by httpserver, and your case is something which could be implemented in that, so it is great you found it. :)

There could be also a function returning a handler function thing. Here is some example:

def respond_with_delay(secs: float, **kwargs):
    def handler(_):
        time.sleep(secs)
        return werkzeug.Response(**kwargs)
    return handler

Then you could use this for the respond_with_handler() call:

httpserver.expect_request("/foobar").respond_with_handler(respond_with_delay(10))

This could also be implemented in a class, but your request to include this to the library also makes sense, I just want to check how the API would look like with it. I need some time to think about this.

[csernazs]

I'm thinking about the implementation. This can be made in various ways.

One idea is that I could add a new method. So it would look like this:

httpserver.expect_request("/permanent").respond_with_data("OK permanent").fault_with(sleep=12)

While I do see additonal value in random sleep, I don't want to add that functionality to pytest-httpserver, so it would be either a callable instead of the int, something like this:

import random

def my_random_sleep():
    return random.randint(5, 10)

httpserver.expect_request("/foo").respond_with_data("Foo").fault_with(sleep=my_random_sleep)

Or it could be an iterator:

import random

def my_random_sleep():
    yield random.randint(5, 10)

httpserver.expect_request("/foo").respond_with_data("Foo").fault_with(sleep=my_random_sleep)

Then I thought that this line is getting bigger and bigger, and fault "specifications" could be re-usable also.

So it could also be like:

import time

class Fault:
    def __init__(self, sleep_seconds: float | None = None):
        self.seconds = sleep_seconds

    def sleep(self):
        time.sleep(self.seconds)

    httpserver.expect_request("/foo").respond_with_data("Foo", fault=Fault(5)) # Fault(5) could also be assigned to a variable

In this way, advanced (random) sleep could be added by subclassing, with the added abstract base class (which would not define the __init__).

Also, regarding the other parameters such as chunked response or random garbage, this could either go to the Fault class (or a new paramater to the fault_with method). Or there could be a super-generic super complicated composable fault objects - I'm sure I want to do something simple but also having a Fault class which can sleep and generate random data and do various other faults is clearly a breaking of SRP.

What do you think? Do you have any concrete example for the usage?

[adamdougal]

Heya, thanks for the amount of time you've spent looking at this!

Your final suggestion sounds good to me. Though perhaps you could consider the Fault class as abstract and have various implementations e.g. DelayFault, ChunkedResponseFault etc. The only thing with that I guess is that some faults will just do actions and succeed e.g. a delay and some will need to return something faulty, I'm not sure if that is easily possible? I'm more of a Java/Golang dev and somewhat new to Python so not sure what the best way to structure this is.

[csernazs]

Hi @adamdougal ,

I thought that faults are actually hooks, so it is not limited to making faulty responses but can also be used to alter the response in any way.

I made the POC implementation in #295 , could you please take a look?

I implemented the Delay and the Garbage hooks but stuggling a bit with the chunked encoding one (but I'll be able to do it I'm sure). There's also a Chain hook which calls the specified hooks one by one.

Each hook is a callable, a python function which rececives the request and the response object, and should return a new response object which will be used later (Delay does not change the response objects).

What do you think?

[adamdougal]

Heya @csernazs,

Apologies for the delay, I've been away on holiday.

Great idea to implement it this way! I can already see this being useful in other ways, such as adding auto generated HMAC signatures to responses.

Is there any reason to only allow one hook? In the context of faults/delays, only one makes sense. But thinking of other use cases, e.g. HMAC it could be useful to be able to chain multiple. This can always come later though.

Thanks again!

[csernazs]

hi @adamdougal ,

No worries, I was also on holiday. :)

I was thinking about supporting multiple hooks, maybe I will update the API to receive a sequence of hooks, but I also think that if the list is mutable then inserting or manipulating the order of the hooks could lead to hard to debug errors so at the moment I definitely don't want to add like add_hook or something like that (which would add a new hook to the list). In such case the new hook would have no idea what kind of response it will receive.

I think if there's a Chain hook what I implemented, that could make the API simple at "my" end (so pytest-httpserver need to worry about one hook, and that's it), on the other hand, it is a bit unconfortable for the user.

I don't want to re-implement flask (I'm not saying having multiple hooks would re-implement it, but it will move the project to that direction, IMO). pytest-httpserver should be a low-level, extensible http server.

Let me sleep on it.

Zsolt

[csernazs]

@adamdougal

Hi there,

I've released 1.0.11 with the features you suggested, so hooks API and simulating faults have been released. https://pypi.org/project/pytest_httpserver/

These have been documented in the howto: https://pytest-httpserver.readthedocs.io/en/latest/howto.html#adding-side-effects

https://pytest-httpserver.readthedocs.io/en/latest/howto.html#querying-the-log

Also, I'm sorry for the delayed release. I promise next time it will be quicker :)

Thanks again for your suggestions!

Zsolt

ps: I'm closing your issues, but feel free to re-open them if you spot any issues.

[adamdougal]

Heya Zsolt, that's great news, thanks!