Deployment tests interfere after upgrading from Quarkus 3.5.0.CR1 to 3.5.0

[ppalaga]

Describe the bug

After upgrading from Quarkus 3.5.0.CR1 to 3.5.0, I can see some deployment tests failing in Quarkus CXF core deployment module, but they still pass when I run them in isolation.

Steps to reproduce:

git clone git@github.com:ppalaga/quarkus-cxf.git -o ppalaga
cd quarkus-cxf
git fetch ppalaga q36676
git checkout q36676
mvnd clean install -DskipTests -Dquarkus.build.skip
cd core/deployment
mvnd clean test
...
[ERROR] ServerDevModeTest
...
[ERROR]   IncludesConflictJava2WsTest
...
[ERROR]   IncludesExcludesJava2WsTest
...
[ERROR]   WsdlNameTemplateJava2WsTest
...
[ERROR] Tests run: 49, Failures: 1, Errors: 3, Skipped: 1

But when the failing tests are run individually, they pass. E.g.

mvnd clean test -Dtest=WsdlNameTemplateJava2WsTest
...
[INFO] BUILD SUCCESS

[ppalaga]

Were there any changes in deployment test class loading between 3.5.0.CR1 to 3.5.0?

[ppalaga]

I can see a similar issue in a @QuarkusTest using a @TestProfile:

cd integration-tests/server
mvnd clean test 
...
[ERROR]   XForwardedHeadersTest.testXForwardedHostHeader
...

But again, when I run the same test in isolation, it is passing:

mvnd clean test -Dtest=XForwardedHeadersTest
...
[INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0
...
[INFO] BUILD SUCCESS
...

I do not see any apparent suspect between 3.5.1.CR1 and 3.5.0

May be this one? https://github.com/quarkusio/quarkus/commit/91cddc603fcee915d328f3b0c0460a1eb2e9c773 @radcortez could the above change have the described impact?

[ppalaga]

May be this one? 91cddc6 @radcortez could the above change have the described impact?

No, it is not the cause of this problem. I have built quarkus 3.5.0 without that change and I still see the same failures.

[ppalaga]

I have found what is causing this issue: In the branch, where I see the failures I had a change that workarounds wrong resource precedence in native mode through setting some runtime property via SystemPropertyBuildItem:

https://github.com/ppalaga/quarkus-cxf/commit/a5d70250b849c7c3619e8acedc7329a4dc91dda3#diff-75da9f7e1c3e1384976a49d0ad67baf36638a284b5c6d1beafa5846d6ff55fdfR184-R188

It looks like this change breaks the test isolation.

@radcortez is it a known problem that SystemPropertyBuildItem can break the test isolation?

When I change it to NativeImageSystemPropertyBuildItem (which still works as a workaround for the other issue), the deployment and JVM tests work as expected. When I change the build step only to execute when the build target is native (which still works as a workaround for the other issue), the deployment and JVM tests work as expected.

[radcortez]

@radcortez is it a known problem that SystemPropertyBuildItem can break the test isolation?

I guess it will depend on what the configuration that you set is doing. This will set whatever config from the build step, directly into SystemProperties in Quarkus main.

[radcortez]

@ppalaga is this still an issue?

[ppalaga]

Yep, if I remove `` from

https://github.com/quarkiverse/quarkus-cxf/blob/0eb4bae0cbf30883983b3b4f3df345338b0acf65/extensions/core/deployment/src/main/java/io/quarkiverse/cxf/deployment/QuarkusCxfProcessor.java#L192

Then the same tests plus one new are failing

[radcortez]

I noticed that a new test was added after this issue that fails consistently (either when running in the entire test suite or standalone): io.quarkiverse.cxf.deployment.test.dev.DevUiTest. The error is exactly the same, which makes the other tests fail. I could pinpoint a null entry in a extra.class property to an apparent ClassNotFoundException in org.apache.cxf.jaxws.spi.WrapperClassLoader#createWrapperClass for io.quarkiverse.cxf.deployment.test.dev.jaxws_asm.GetService.

java.lang.NullPointerException: Cannot invoke "java.lang.Class.getName()" because "clazz" is null

    at org.apache.cxf.common.util.PackageUtils.getPackageName(PackageUtils.java:47)
    at org.apache.cxf.common.jaxb.JAXBUtils.scanPackages(JAXBUtils.java:865)
    at org.apache.cxf.common.jaxb.JAXBUtils.scanPackages(JAXBUtils.java:843)
    at org.apache.cxf.common.jaxb.JAXBContextCache.scanPackages(JAXBContextCache.java:153)
    at org.apache.cxf.jaxb.JAXBDataBinding.createJAXBContextAndSchemas(JAXBDataBinding.java:495)
    at org.apache.cxf.jaxb.JAXBDataBinding.initialize(JAXBDataBinding.java:358)
    at org.apache.cxf.service.factory.AbstractServiceFactoryBean.initializeDataBindings(AbstractServiceFactoryBean.java:87)
    at org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean.buildServiceFromWSDL(ReflectionServiceFactoryBean.java:425)
    at org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean.initializeServiceModel(ReflectionServiceFactoryBean.java:527)
    at org.apache.cxf.wsdl.service.factory.ReflectionServiceFactoryBean.create(ReflectionServiceFactoryBean.java:262)
    at org.apache.cxf.jaxws.support.JaxWsServiceFactoryBean.create(JaxWsServiceFactoryBean.java:199)
    at org.apache.cxf.jaxws.ServiceImpl.createPort(ServiceImpl.java:466)
    at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:342)
    at org.apache.cxf.jaxws.ServiceImpl.getPort(ServiceImpl.java:337)
    at jakarta.xml.ws.Service.getPort(Service.java:210)
    at io.quarkiverse.cxf.deployment.test.dev.DevUiTest.getClient(DevUiTest.java:79)
    at io.quarkiverse.cxf.deployment.test.dev.DevUiTest.clientsAndServices(DevUiTest.java:41)
    at java.base/java.lang.reflect.Method.invoke(Method.java:568)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

I couldn't find any indication that setting the config breaks the test isolation. The config is applied to every test, but I can't explain why the tests break either. I suspect that maybe something else is leaking.