search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.87k stars 2.71k forks source link

Use dev services by default in Quarkus' own integration tests #44124

Open yrodiere opened 1 month ago

yrodiere commented 1 month ago

Description

Test modules in integration-tests rely on dedicated Maven configuration to start containers of services used in tests.

Could we not just rely on dev services for most of them?

One benefit would be additional testing of dev services in scenarios that are as close as possible to the "real world" -- as opposed to most dev service tests we have right now, which rely on QuarkusDevModeTest -- which, as great as it is, is still a simulated setup.

Another benefit in our own dev environments would be that a test failure in surefire wouldn't leave the containers running, forcing us to manually kill them before restarting the tests (I swear I had to do this many times).

A downside could be performance, as we would start the containers twice per integration test module (once for surefire tests, once for failsafe tests) instead of once. That can be worked around on CI by enabling container reuse, though then cleanup mid-build will be challenging.

Related: #43980. Which could, perhaps, have been caught if we used dev services in our integration tests. But I think the points above stand even if it's a bad example.

Implementation ideas

No response

quarkus-bot[bot] commented 1 month ago

/cc @geoand (devservices), @stuartwdouglas (devservices)

yrodiere commented 1 month ago

@stuartwdouglas (devservices)

We might want to update the bot config...

geoand commented 1 month ago

Test modules in integration-tests rely on dedicated Maven configuration to start containers of services used in tests.

For a few things there might be specific reasons why Testcontainers were not used, but I believe that most cases are just leftovers from when we didn't have DevServices

gsmet commented 1 month ago

Yeah, same here. For MongoDB, I remember we have to avoid Dev Services because we are using clustering features. But apart from that, I think we could move to Dev Services.

gsmet commented 1 month ago

Note that there's one important limitation though: Windows CI doesn't support containers. So that means we need to keep not enabling the tests and not starting dev services when containers are not enabled.

yrodiere commented 1 month ago

Note that there's one important limitation though: Windows CI doesn't support containers. So that means we need to keep not enabling the tests and not starting dev services when containers are not enabled.

Looks like an argument in favor of dev services to me: we'll need to disable the tests (same as today), but now disabling the tests will automatically disable container usage, since dev services only start when you start the tests :)

holly-cummins commented 1 week ago

I don't know the background of why we weren't using dev services, so can't comment with authority, but I agree that having less home-rolled code in our test poms, in combination with more coverage of dev services, can only be a good thing. For things like catching the next https://github.com/quarkusio/quarkus/issues/43980 we'd also probably need a regular Mac CI, but that's also something we want to do anyway.

gsmet commented 1 week ago

I don't know the background of why we weren't using dev services

There is a mix of they weren't available at that time or flexible enough and I think in one case (some MongoDB testing), we actually still need clustering support that is easier to handler when starting MongoDB manually.

So I think it's worth doing when actually doable.