search

lipanski / mockito

HTTP mocking for Rust!
MIT License
695 stars 59 forks source link

Mocks can receive requests from previous tests #140

Closed Matthias247 closed 1 year ago

Matthias247 commented 3 years ago

I observed an issue where one of my tests using this library failed sporadically with

Expected 2 request(s) to: GET /path ...but received 3

The mock in this test is set up using:

let m = mock("GET", "/path")
    .with_status(500)
    ... // add header and body
    .create()
    .expect(2);

and the client trying to use the mock in this test definitely performs only 2 requests.

After adding a variety of additional logs and a custom header per test, I noticed that the mock received and processed a request from a previous test. That incremented the request count and lead the test to fail.

My understanding is that this connection attempt from the previous test lead to a socket with associated data still sitting in the backlog, and that isn't immediately cleared even if one test closes the listener socket. So the next listener which is bound to the same port ( 1234) will accept that connection and read the data.

The mutex which attempts to isolate test cases against each other doesn't help here. Adding some wait time after each test does, but slows things down even further.

I tried modifying mockito to always bind to :0 and let the OS choose a free port. However it still picked the same for all tests and run into the issue.

So the main thing to make this more reliable besides avoiding expectation around the number of requests seems the following: When the mock is created, accept() all connections which are pending on the socket and drop them before returning to the user. However it seems like there is no try_accept() method on the synchronous listener available.

fermanjj commented 2 years ago

I looks like you're calling .expect() after you create the mock. Does this same issue occur if you swap it to chaining .expect() before calling .create()?

lipanski commented 2 years ago

I tried modifying mockito to always bind to :0 and let the OS choose a free port. However it still picked the same for all tests and run into the issue.

yes, currently Mockito runs all tests on the same server and this comes with various limitations. I've been wanting to move away from this model and allow (at least) one server per test. This would also mean you can start a mock server explicitly, making it possible to simulate multiple "hosts" within the same test. Didn't get to it yet and not sure when I'll be able to squeeze it in, but it should solve your issue.

Until that happens, if you have a quicker solution, PRs are more than welcome.

lipanski commented 1 year ago

This might be fixed in https://github.com/lipanski/mockito/releases/tag/0.32.0