search

weld / weld-testing

Set of test framework extensions (JUnit 4, JUnit 5, Spock) to enhance the testing of CDI components via Weld. Supports Weld 5.
http://weld.cdi-spec.org/
Apache License 2.0
101 stars 30 forks source link

Unable to run embedded web server in unit test due to conflicting weld-se / weld-servlet versions #214

Closed jansohn closed 1 week ago

jansohn commented 2 weeks ago

I'm trying to run an embedded web server (tomcat or jetty) in a junit test. The problem is that during weld bootstrapping when starting the embedded web server it picks up jakarta.enterprise.inject.spi.Extension from the weld-se package (org.jboss.weld.environment.se.WeldSEBeanRegistrant) instead of the weld-servlet package (org.jboss.weld.module.web.WeldWebModule) which leads to the following exception:

Caused by: java.lang.RuntimeException: Service class org.jboss.weld.environment.se.WeldSEBeanRegistrant didn't implement the required interface
        at org.jboss.weld.util.ServiceLoader.loadClass(ServiceLoader.java:230)
        at org.jboss.weld.util.ServiceLoader.loadService(ServiceLoader.java:210)
        at org.jboss.weld.util.ServiceLoader.loadServiceFile(ServiceLoader.java:184)
        at org.jboss.weld.util.ServiceLoader.reload(ServiceLoader.java:164)
        at org.jboss.weld.util.ServiceLoader.iterator(ServiceLoader.java:288)
        at org.jboss.weld.util.collections.ImmutableSet$BuilderImpl.addAll(ImmutableSet.java:158)
        at org.jboss.weld.environment.servlet.WeldServletLifecycle.createDeployment(WeldServletLifecycle.java:275)
        at org.jboss.weld.environment.servlet.WeldServletLifecycle.initialize(WeldServletLifecycle.java:153)
        at org.jboss.weld.environment.servlet.EnhancedListener.onStartup(EnhancedListener.java:66)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4412)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        ... 25 more

Reproducer: https://github.com/jansohn/weld-junit5-embedded-jetty/tree/embedded-tomcat-weld-junit5

Is there any clever trick to make weld-se and weld-servlet-core on the same classloader behave correctly?

manovotn commented 2 weeks ago

What's the use case for mixing weld-junit and jetty together?

Weld-junit is made for SE testing and hence contains Weld SE artifacts (and starts Weld SE container) whereas for servlets, you'd want the Weld Servlet components with a listener starting Weld container instead.

Is there any clever trick to make weld-se and weld-servlet-core on the same classloader behave correctly?

Actually, They don't seem to be on the same classloader here which is the problem. WeldSEBeanRegistrant definitely does implement the interface it complains about - jakarta.enterprise.inject.spi.Extension. And if you take a look at the service loader code and what class loaders are used, you'll find that the jakarta.enterprise.inject.spi.Extensiontype (the expected type we cast to; comes from CDI API artifact) is loaded by ParallelWebappClassLoader whereas Weld SE classes (WeldSeBeanRegistrant) are loaded by jdk.internal.loader.ClassLoaders$AppClassLoader. And this IMO causes the problem.

jansohn commented 2 weeks ago

What's the use case for mixing weld-junit and jetty together?

Weld-junit is made for SE testing and hence contains Weld SE artifacts (and starts Weld SE container) whereas for servlets, you'd want the Weld Servlet components with a listener starting Weld container instead.

We use weld-junit (in test Maven scope) to have @Inject support in our unit tests. The project is a web application so it needs weld-servlet as we are using servlet containers to run our application, e.g. Tomcat.

For integration tests, e.g. high load scenarios, we want to spin up an embedded web server hosting our application which we can use to run the tests against.

In our old javax-style environment we have this running properly with embedded Jetty because embedded Tomcat wasn't working correctly when we set this up years ago.

After switching to jakarta-style environment we are now not able to get it running neither with Jetty (which still seems to have a different issue: https://github.com/jetty/jetty.project/issues/12285) nor Tomcat.

Is there any clever trick to make weld-se and weld-servlet-core on the same classloader behave correctly?

Actually, They don't seem to be on the same classloader here which is the problem. WeldSEBeanRegistrant definitely does implement the interface it complains about - jakarta.enterprise.inject.spi.Extension. And if you take a look at the service loader code and what class loaders are used, you'll find that the jakarta.enterprise.inject.spi.Extensiontype (the expected type we cast to; comes from CDI API artifact) is loaded by ParallelWebappClassLoader whereas Weld SE classes (WeldSeBeanRegistrant) are loaded by jdk.internal.loader.ClassLoaders$AppClassLoader. And this IMO causes the problem.

But actually it shouldn't pick up a class from Weld SE in the first place when it is initialized with org.jboss.weld.environment.servlet.EnhancedListener in a weld-servlet environment, right?

I'm not sure anything can be done in this project about the problem besides offering a weld-servlet mode which apparently does not exist.

I've opened an issue on the Weld issue tracker (https://issues.redhat.com/browse/WELD-2797) because in the end I feel this needs to be handled properly in the initialization/bootstrap code.

manovotn commented 2 weeks ago

But actually it shouldn't pick up a class from Weld SE in the first place when it is initialized with org.jboss.weld.environment.servlet.EnhancedListener in a weld-servlet environment, right?

Weld bootstrap performs a service loader discovery and you have Weld SE in classpath which, I assume, finds the extension in question.

I'm not sure anything can be done in this project about the problem besides offering a weld-servlet mode which apparently does not exist. I've opened an issue on the Weld issue tracker (https://issues.redhat.com/browse/WELD-2797) because in the end I feel this needs to be handled properly in the initialization/bootstrap code.

I don't think this is a supported scenario to be fair. I know servlet (and EE server) testing was available with Arquillian where you can perform test injection as well; did you give that a go?

Here are some thoughts and ideas if you want to dig deeper into this but note that as there is a CL problem, it will likely be more difficult than this:

jansohn commented 2 weeks ago

I don't think this is a supported scenario to be fair. I know servlet (and EE server) testing was available with Arquillian where you can perform test injection as well; did you give that a go?

No, so far I have no experience with Arquillian. Back in the days we started with cdi-unit but it did not support Jakarta EE / JUnit 5 for a long time so we switched to weld-junit5.

Here are some thoughts and ideas if you want to dig deeper into this but note that as there is a CL problem, it will likely be more difficult than this:

  • I recall we had a weld-se-servlet coop test somewhere with Tomcat that approached it the other way around - by starting Weld SE and then skipping bootstrap for servlet.

  • In your reproducer you are currently bootstrapping servlet first - maybe try swapping that around?

    • Weld-junit is by default bootstrapping per-test, but you could try to perform that per-class by using @TestInstance(Lifecycle.PER_CLASS). In such a case, the SE container might be used first and the servlet would just re-use it.

I did try to combine your two examples (posted the code here: https://github.com/jansohn/weld-junit5-embedded-jetty/tree/embedded-tomcat-weld-junit5-nobootstrap). Unfortunately as you expected it also fails:

2024-09-24 08:19:05,394 INFO  [main] org.jboss.weld.Version: WELD-000900: 5.1.3 (Final)
2024-09-24 08:19:05,652 INFO  [main] org.jboss.weld.Bootstrap: WELD-000101: Transactional services not available. Injection of @Inject UserTransaction not available. Transactional observers will be invoked synchronously.
2024-09-24 08:19:05,782 INFO  [main] org.jboss.weld.Event: WELD-000411: Observer method [BackedAnnotatedMethod] org.jboss.weld.junit5.auto.TestInstanceInjectionExtension.rewriteTestClassScope(@Observes ProcessAnnotatedType<T>) receives events for all annotated types. Consider restricting events using @WithAnnotations or a generic type with bounds.
2024-09-24 08:19:06,024 INFO  [main] org.jboss.weld.Bootstrap: WELD-ENV-002003: Weld SE container 85663e00-39c4-4cbc-aac7-4f3eea7f9653 initialized
2024-09-24 08:19:06,040 INFO  [main] com.test.EmbeddedJettyIT: Starting embedded server...
Sept 24, 2024 8:19:06 AM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-nio-auto-1"]
Sept 24, 2024 8:19:06 AM org.apache.catalina.core.StandardService startInternal
INFO: Starting service [Tomcat]
Sept 24, 2024 8:19:06 AM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet engine: [Apache Tomcat/10.1.30]
Sept 24, 2024 8:19:06 AM org.apache.catalina.startup.ContextConfig getDefaultWebXmlFragment
INFO: No global web.xml found
2024-09-24 08:19:09,057 INFO  [main] org.jboss.weld.environment.servletWeldServlet: WELD-ENV-001008: Initialize Weld using ServletContainerInitializer
[...]
Caused by: java.lang.ClassCastException: class org.jboss.weld.bean.builtin.BeanManagerProxy cannot be cast to class org.jboss.weld.manager.api.WeldManager (org.jboss.weld.bean.builtin.BeanManagerProxy is in unnamed module of loader 'app'; org.jboss.weld.manager.api.WeldManager is in unnamed module of loader org.apache.catalina.loader.ParallelWebappClassLoader @1a4d1ab7)
        at org.jboss.weld.environment.servlet.WeldServletLifecycle.initialize(WeldServletLifecycle.java:126)
        at org.jboss.weld.environment.servlet.EnhancedListener.onStartup(EnhancedListener.java:66)
        at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:4412)
        at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:164)
        ... 82 more
jansohn commented 2 weeks ago

Out of curiosity: I'd expect that linked test to fail in the same way as my code does. Can you check that?

manovotn commented 2 weeks ago

Out of curiosity: I'd expect that linked test to fail in the same way as my code does. Can you check that?

That one passes; the CI is running it on PRs, see this command for how to run it manually. Note that it does not use weld-junit and runs on tomcat which means that the order in which containers start is reversed (Weld SE first, then servlet which discovers an already running container), plus tomcat will have different class loader setup, potentially avoiding this issue.

jansohn commented 2 weeks ago

Out of curiosity: I'd expect that linked test to fail in the same way as my code does. Can you check that?

That one passes; the CI is running it on PRs, see this command for how to run it manually. Note that it does not use weld-junit and runs on tomcat which means that the order in which containers start is reversed (Weld SE first, then servlet which discovers an already running container), plus tomcat will have different class loader setup, potentially avoiding this issue.

Thanks, I was able to successfully run this locally on my machine, too. I'm still trying to figure out though why it is not working with my minimal example. I can see that there are arquillian dependencies and configuration files in place. But when I step through it in debug mode it does not even initialize anything from arquillian, so I don't think that is the difference. I'll play around with it some more to see if I can figure it out...

manovotn commented 2 weeks ago

Arquillian is not present in that test. I suggested that framework because I know there are Arquillian adapters for servlets (such as this one for Tomcat) and those allow you to boot the servlet, use Weld and have tests injected all at the same time. From your description that seems to fit better to what you expect than mixing SE and servlet; give it a look.

jansohn commented 2 weeks ago

I finally seem to have a working solution: https://github.com/jansohn/weld-junit5-embedded-jetty/blob/embedded-tomcat-weld-junit5/src/test/java/com/test/EmbeddedJettyIT.java

The only draw-back is that I cannot use @Inject on the test classes, otherwise the Weld initialization in the Tomcat container blows up.

The main issues were:

manovotn commented 2 weeks ago

weld-junit and Tomcat used the same Weld instance which caused problems so I decided to pass an explicit WELD_CONTEXT_ID_KEY to the servlet container to make it use its own Weld instance exclusively

I think I don't understand the use case still. You end up with two different Weld containers that know nothing about one another - what's the point/gain? You won't have access to beans from the other container which seems to defeat the purpose of injecting into the test class, doesn't it?

jansohn commented 1 week ago

weld-junit and Tomcat used the same Weld instance which caused problems so I decided to pass an explicit WELD_CONTEXT_ID_KEY to the servlet container to make it use its own Weld instance exclusively

I think I don't understand the use case still. You end up with two different Weld containers that know nothing about one another - what's the point/gain? You won't have access to beans from the other container which seems to defeat the purpose of injecting into the test class, doesn't it?

To be honest in my use case it does not make a difference, I just want to spin up my web application with CDI support to hit some servlets simultaneously for example. For preparation I sometimes need to inject in the test classes, too but it doesn't have to be the same instance as the one running in the web server.

Unfortunately my "solution" does not work once I add some more complexity to it (JSF, OmniFaces, etc.) so I'm pretty much back to square one...

manovotn commented 1 week ago

I am sorry but this is out of scope of what Weld-junit is designed for.

I really recommend taking a look at Arquillian which covers this a lot better and I think most of the server/servlet testing is done with it anyway. Or perhaps peek into testsuites of those servlets to see what they leverage and how.

jansohn commented 1 week ago

I decided to check out Arquillian today (https://github.com/jansohn/weld-junit5-embedded-jetty/tree/arquillian-embedded-tomcat) and it suffers from the exact same issue as all the other methods I have tried.

Caused by: java.lang.RuntimeException: Service class org.jboss.weld.environment.se.WeldSEBeanRegistrant didn't implement the required interface

There is no classpath isolation at all in Arquillian as it seems: https://issues.redhat.com/browse/ARQ-1946

I don't think anything needs to be fixed here in weld-testing, but I hope you can reconsider offering an option to specify which weld flavor to use during initialization if both (se and servlet) are on the same classpath.

manovotn commented 1 week ago

I decided to check out Arquillian today (https://github.com/jansohn/weld-junit5-embedded-jetty/tree/arquillian-embedded-tomcat) and it suffers from the exact same issue as all the other methods I have tried.

You shouldn't need Weld SE in that test case scenario; it's creeping in from somewhere - probably your Weld-junit dep (which is also redundant when running Arq container).

With that out of the way, I am getting:

[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0

Here's the PR - https://github.com/jansohn/weld-junit5-embedded-jetty/pull/1

jansohn commented 1 week ago

Of course, I know that but I want to be able to use weld-junit in other test classes of my project...