lipanski / mockito

HTTP mocking for Rust!
MIT License
695 stars 59 forks source link

parallel tests each creating an async server stomp #200

Closed evbo closed 7 months ago

evbo commented 8 months ago

It seems if my tests run in parallel and both define a ServerGuard the mocks stomp one another.

The only solution I've found is running them serially.

How can multiple tests create a server like?:

        let mut server = mockito::Server::new_async().await;
        server.mock(..)

without affecting other tests?

lipanski commented 8 months ago

@evbo can you provide some code to reproduce this? this repo runs tests in parallel and creates lots of server guards without any knows issues so far.

lipanski commented 7 months ago

Also note that there's a limit to the amount of servers you can run in parallel (20 for mac, 50 otherwise) and if you want more you should just use Server::new_with_opts, which bypasses the server pool.

evbo commented 7 months ago

Sorry I couldn't get to this in time. I will try new_with_opts as my tests are definitely stomping and I only have 2 servers! May be a while when I get around to it... Like I said everything works great running tests one at a time at least.

evbo commented 5 months ago

new_with_opts_async makes no difference. This issue continually occurs on projects where I am testing using tokio-test

I suggest leaving it open. It's real. Sorry I can't post my code, but here's a dummy version:

#[cfg(test)]
mod tests {
    use mockito::Matcher;
    use mockito::ServerOpts;
    use serde::Serialize;
    use std::{env, str::FromStr};
    use tokio::sync::AcquireError;
    use tokio_test as _;

    use super::super::*;

    #[tokio::test]
    async fn train1_test() {
        let (client, _server) = mock().await;
        assert!(..);
    }

    #[tokio::test]
    async fn train2_test() {
        let (client, _server) = mock().await;
        assert!(..);
    }

    async fn mock() -> (reqwest::Client, mockito::Server) {

        let mut server = mockito::Server::new_with_opts_async(ServerOpts::default()).await;
        let client = http_client();

        server
            .mock("GET", "/base/logic")
            .with_status(200)
            .create();

        server
            .mock("POST", "/base/logic")
            .with_status(200)
            .create();

        server
            .mock("GET", &*format!("/base/logic/{id}/saved"))
            .with_status(200)
            .create();

        server
            .mock(
                "POST",
                Matcher::Regex(format!("^/base/logic/{id}/saves/[^/]+$")),
            )
            .with_status(200)
            .with_body("{}")
            .create();

        server
            .mock("POST", &*format!("/base/logic/{id}/save"))
            .with_status(200)
            .with_body("{}")
            .create();

        server
            .mock("GET", &*format!("/base/logic/{id}"))
            .with_status(200)
            .create();

        (client, server)
    }
}
lipanski commented 5 months ago

@evbo you should use the async methods if you’re going async: create_async().await instead of create. see the async section of the docs for reference. let me know if that fixes your issue.

evbo commented 5 months ago

Thank you @lipanski but it makes no difference. Not only that, if I println! my code judiciously I can see the _server is being shared between tests, such that it has values from prior tests in the mocked paths.

So you definitely have a bug stemming from some kind of shared mutability between tests. I highly recommend reopening this issue.

lipanski commented 5 months ago

@evbo can you share some code to reproduce this? I can't really leave an issue open if I can't reproduce the issue. might be on your side, might be our side - but we can only know once you share some more information. maybe simplify your logic to the point where it's just mockito code and some basic abstraction of what you're doing, including your println! stuff as long as it compiles. could also be a good exercise to track down any potential issues on your end.

weinberggithub commented 2 weeks ago

same problem, with log Mock not found