search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.36k stars 2.56k forks source link

Class Loader still used while it's supposed to be closed #41233

Open gsmet opened 3 weeks ago

gsmet commented 3 weeks ago

As part of my work trying to improve the CL situation, I stumbled upon a few cases of $TITLE for which I would need some help.

In my patch, I introduce a safe guard throwing an exception when you are trying to load a class from a CL that has been previously closed. The rationale is that we shouldn't load any class from a CL that has been closed.

I already fixed a few of these problems... but I still have some going on that are a lot less easy to pinpoint.

These issues where all observed in this specific CI job run: https://github.com/gsmet/quarkus/actions/runs/9535653448/job/26281846636

  1. io.vertx.core.impl.VertxImpl$1.operationComplete
2024-06-16 12:05:59,044 WARN [io.net.uti.con.DefaultPromise] (globalEventExecutor-1-1) An exception was thrown by io.vertx.core.impl.VertxImpl$1.operationComplete(): java.lang.IllegalStateException: Class loader Augmentation Class Loader: PROD for minikube-with-defaults has been closed and may not be accessed anymore
at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen(QuarkusClassLoader.java:723)
at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:499)
at io.vertx.core.impl.VertxImpl$1.operationComplete(VertxImpl.java:986)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492)
at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:97)
at io.netty.util.concurrent.MultithreadEventExecutorGroup$1.operationComplete(MultithreadEventExecutorGroup.java:117)
at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590)
at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557)
at io.netty.util.concurrent.DefaultPromise.access$200(DefaultPromise.java:35)
at io.netty.util.concurrent.DefaultPromise$1.run(DefaultPromise.java:503)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)
at io.netty.util.concurrent.GlobalEventExecutor$TaskRunner.run(GlobalEventExecutor.java:262)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)

This one is due to the delay to handle the graceful shutdown:

From io.vertx.core.impl.VertxImpl:

       eventLoopGroup.shutdownGracefully(0, 10, TimeUnit.SECONDS).addListener(new GenericFutureListener() {
            @Override
            public void operationComplete(io.netty.util.concurrent.Future future) throws Exception {
              if (!future.isSuccess()) {
                log.warn("Failure in shutting down event loop group", future.cause());
              }
              if (metrics != null) {
                metrics.close();
              }
              if (tracer != null) {
                tracer.close();
              }

              checker.close();

              if (completionHandler != null) {
                eventLoopThreadFactory.newThread(() -> {
                  completionHandler.handle(Future.succeededFuture());
                }).start();
              }
            }
          });

A more complete stacktrace:

[INFO] java.lang.IllegalArgumentException: io.vertx.core.impl
[INFO]     at java.lang.ClassLoader.definePackage (ClassLoader.java:2215)
[INFO]     at io.quarkus.bootstrap.classloading.QuarkusClassLoader.definePackage (QuarkusClassLoader.java:576)
[INFO]     at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass (QuarkusClassLoader.java:542)
[INFO]     at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass (QuarkusClassLoader.java:501)
[INFO]     at java.lang.ClassLoader.defineClass1 (Native Method)
[INFO]     at java.lang.ClassLoader.defineClass (ClassLoader.java:1017)
[INFO]     at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass (QuarkusClassLoader.java:543)
[INFO]     at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass (QuarkusClassLoader.java:501)
[INFO]     at io.vertx.core.impl.TaskQueue.execute (TaskQueue.java:157)
[INFO]     at io.vertx.core.impl.ContextImpl.internalExecuteBlocking (ContextImpl.java:217)
[INFO]     at io.vertx.core.impl.ContextImpl.executeBlocking (ContextImpl.java:189)
[INFO]     at io.vertx.core.impl.ContextImpl.executeBlockingInternal (ContextImpl.java:111)
[INFO]     at io.vertx.core.impl.ContextInternal.executeBlockingInternal (ContextInternal.java:152)
[INFO]     at io.vertx.core.impl.VertxInternal.executeBlockingInternal (VertxInternal.java:196)
[INFO]     at io.vertx.core.impl.VertxImpl.deleteCacheDirAndShutdown (VertxImpl.java:959)
[INFO]     at io.vertx.core.impl.VertxImpl.lambda$null$11 (VertxImpl.java:747)
[INFO]     at io.vertx.core.impl.VertxImpl.lambda$closeClusterManager$9 (VertxImpl.java:713)
[INFO]     at io.vertx.core.impl.future.FutureImpl$4.onSuccess (FutureImpl.java:176)
[INFO]     at io.vertx.core.impl.future.FutureBase.emitSuccess (FutureBase.java:66)
[INFO]     at io.vertx.core.impl.future.FutureImpl.addListener (FutureImpl.java:231)
[INFO]     at io.vertx.core.impl.future.FutureImpl.onComplete (FutureImpl.java:199)
[INFO]     at io.vertx.core.impl.VertxImpl.closeClusterManager (VertxImpl.java:708)
[INFO]     at io.vertx.core.impl.VertxImpl.lambda$null$12 (VertxImpl.java:745)
[INFO]     at io.vertx.core.impl.future.FutureImpl$4.onSuccess (FutureImpl.java:176)
[INFO]     at io.vertx.core.impl.future.FutureBase.emitSuccess (FutureBase.java:66)
[INFO]     at io.vertx.core.impl.future.FutureImpl.addListener (FutureImpl.java:231)
[INFO]     at io.vertx.core.impl.future.FutureImpl.onComplete (FutureImpl.java:199)
[INFO]     at io.vertx.core.impl.VertxImpl.lambda$null$13 (VertxImpl.java:744)
[INFO]     at io.vertx.core.impl.resolver.DefaultResolverProvider.close (DefaultResolverProvider.java:34)
[INFO]     at io.vertx.core.impl.AddressResolver.close (AddressResolver.java:118)
[INFO]     at io.vertx.core.impl.VertxImpl.lambda$null$14 (VertxImpl.java:741)
[INFO]     at io.vertx.core.impl.future.FutureImpl$4.onSuccess (FutureImpl.java:176)
[INFO]     at io.vertx.core.impl.future.FutureBase.emitSuccess (FutureBase.java:66)
[INFO]     at io.vertx.core.impl.future.FutureImpl.addListener (FutureImpl.java:231)
[INFO]     at io.vertx.core.impl.future.FutureImpl.onComplete (FutureImpl.java:199)
[INFO]     at io.vertx.core.impl.VertxImpl.lambda$null$15 (VertxImpl.java:740)
[INFO]     at io.vertx.core.impl.ContextImpl.emit (ContextImpl.java:328)
[INFO]     at io.vertx.core.impl.ContextImpl.lambda$emit$5 (ContextImpl.java:335)
[INFO]     at io.netty.util.concurrent.AbstractEventExecutor.runTask (AbstractEventExecutor.java:173)
[INFO]     at io.netty.util.concurrent.AbstractEventExecutor.safeExecute (AbstractEventExecutor.java:166)
[INFO]     at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks (SingleThreadEventExecutor.java:469)
[INFO]     at io.netty.channel.nio.NioEventLoop.run (NioEventLoop.java:569)
[INFO]     at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:994)
[INFO]     at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
[INFO]     at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)
[INFO]     at java.lang.Thread.run (Thread.java:840)
  1. io.quarkus.netty.deployment.JBossNettyLoggerFactory$JBossNettyInternalLogger.warn
2024-06-16 12:04:49,958 WARN [io.net.cha.nio.NioEventLoop] (vert.x-eventloop-thread-0) Unexpected exception in the selector loop.: java.lang.IllegalStateException: Class loader Augmentation Class Loader: PROD for metrics has been closed and may not be accessed anymore
at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen(QuarkusClassLoader.java:723)
at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:499)
at io.quarkus.netty.deployment.JBossNettyLoggerFactory$JBossNettyInternalLogger.warn(JBossNettyLoggerFactory.java:199)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:168)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:469)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
  1. io.netty.util.concurrent.GlobalEventExecutor.startThread
[INFO] java.lang.IllegalStateException: Class loader Augmentation Class Loader: PROD for kubernetes-docker-build-and-deploy-statefulset-0.1-SNAPSHOT has been closed and may not be accessed anymore
[INFO] at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen (QuarkusClassLoader.java:723)
[INFO] at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass (QuarkusClassLoader.java:499)
[INFO] at io.netty.util.concurrent.GlobalEventExecutor.startThread (GlobalEventExecutor.java:239)
[INFO] at io.netty.util.concurrent.GlobalEventExecutor.execute0 (GlobalEventExecutor.java:227)
[INFO] at io.netty.util.concurrent.GlobalEventExecutor.execute (GlobalEventExecutor.java:221)
[INFO] at io.netty.util.concurrent.DefaultPromise.safeExecute (DefaultPromise.java:862)
[INFO] at io.netty.util.concurrent.DefaultPromise.notifyListeners (DefaultPromise.java:500)
[INFO] at io.netty.util.concurrent.DefaultPromise.setValue0 (DefaultPromise.java:636)
[INFO] at io.netty.util.concurrent.DefaultPromise.setSuccess0 (DefaultPromise.java:625)
[INFO] at io.netty.util.concurrent.DefaultPromise.setSuccess (DefaultPromise.java:97)
[INFO] at io.netty.util.concurrent.SingleThreadEventExecutor$4.run (SingleThreadEventExecutor.java:1056)
[INFO] at io.netty.util.internal.ThreadExecutorMap$2.run (ThreadExecutorMap.java:74)
[INFO] at io.netty.util.concurrent.FastThreadLocalRunnable.run (FastThreadLocalRunnable.java:30)

And related:

 2024-06-16 12:04:07,527 ERROR [io.net.uti.con.Def.rejectedExecution] (vert.x-acceptor-thread-0) Failed to submit a listener notification task. Event loop shut down?: java.lang.IllegalArgumentException: io.netty.util.concurrent
    at java.base/java.lang.ClassLoader.definePackage(ClassLoader.java:2215)
    at io.quarkus.bootstrap.classloading.QuarkusClassLoader.definePackage(QuarkusClassLoader.java:576)
    at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:542)
    at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:501)
    at io.netty.util.concurrent.GlobalEventExecutor.startThread(GlobalEventExecutor.java:239)
    at io.netty.util.concurrent.GlobalEventExecutor.execute0(GlobalEventExecutor.java:227)
    at io.netty.util.concurrent.GlobalEventExecutor.execute(GlobalEventExecutor.java:221)
    at io.netty.util.concurrent.DefaultPromise.safeExecute(DefaultPromise.java:862)
    at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:500)
    at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636)
    at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625)
    at io.netty.util.concurrent.DefaultPromise.setSuccess(DefaultPromise.java:97)
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:1056)
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
    at java.base/java.lang.Thread.run(Thread.java:840)
  1. io.micrometer.core.instrument.binder.jvm.JvmHeapPressureMetrics
java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for QuarkusTestNested1WithCommonCaseTest (QuarkusTest) has been closed and may not be accessed anymore
at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen(QuarkusClassLoader.java:723)
at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:499)
at io.micrometer.core.instrument.binder.jvm.JvmHeapPressureMetrics.lambda$monitor$3(JvmHeapPressureMetrics.java:116)
at java.management/sun.management.NotificationEmitterSupport.sendNotification(NotificationEmitterSupport.java:155)
at jdk.management/com.sun.management.internal.GarbageCollectorExtImpl.createGCNotification(GarbageCollectorExtImpl.java:115)
  1. Instrumentation reload test
2024-06-16T15:46:48.9763012Z 2024-06-16 15:46:48,905 INFO  [io.qua.dep.dev.fil.StaticFileManager] (vert.x-worker-thread-1) Ignoring module-info.java in dev mode, set the `quarkus.live-reload.ignore-module-info` property to `false` in your project descriptor (`pom.xml` or `build.gradle`) to disable this behavior.
2024-06-16T15:46:49.1767168Z 2024-06-16 15:46:49,081 ERROR [io.qua.dep.dev.RuntimeUpdatesProcessor] (vert.x-worker-thread-1) Failed to replace classes via instrumentation: java.lang.IllegalStateException: Class loader Deployment Class Loader: DEV for acme-1.0-SNAPSHOT has been closed and may not be accessed anymore
2024-06-16T15:46:49.1770682Z    at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen(QuarkusClassLoader.java:727)
2024-06-16T15:46:49.1772859Z    at io.quarkus.bootstrap.classloading.QuarkusClassLoader.getElementsWithResource(QuarkusClassLoader.java:608)
2024-06-16T15:46:49.1775071Z    at io.quarkus.deployment.steps.ClassTransformingBuildStep$1.apply(ClassTransformingBuildStep.java:151)
2024-06-16T15:46:49.1777149Z    at io.quarkus.deployment.steps.ClassTransformingBuildStep$1.apply(ClassTransformingBuildStep.java:126)
2024-06-16T15:46:49.1779287Z    at io.quarkus.deployment.steps.ClassTransformingBuildStep.transform(ClassTransformingBuildStep.java:74)
2024-06-16T15:46:49.1781372Z    at io.quarkus.deployment.dev.IsolatedDevModeMain$2.apply(IsolatedDevModeMain.java:250)
2024-06-16T15:46:49.1782968Z    at io.quarkus.deployment.dev.IsolatedDevModeMain$2.apply(IsolatedDevModeMain.java:247)
2024-06-16T15:46:49.1784749Z    at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:502)
2024-06-16T15:46:49.1786439Z    at io.quarkus.deployment.dev.RuntimeUpdatesProcessor.doScan(RuntimeUpdatesProcessor.java:448)
2024-06-16T15:46:49.1788237Z    at io.quarkus.vertx.http.runtime.devmode.VertxHttpHotReplacementSetup$6.call(VertxHttpHotReplacementSetup.java:161)
2024-06-16T15:46:49.1790223Z    at io.quarkus.vertx.http.runtime.devmode.VertxHttpHotReplacementSetup$6.call(VertxHttpHotReplacementSetup.java:148)
2024-06-16T15:46:49.1792376Z    at io.vertx.core.impl.ContextImpl.lambda$executeBlocking$0(ContextImpl.java:178)
2024-06-16T15:46:49.1793816Z    at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:279)
2024-06-16T15:46:49.1795247Z    at io.vertx.core.impl.ContextImpl.lambda$internalExecuteBlocking$2(ContextImpl.java:210)
2024-06-16T15:46:49.1796526Z    at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18)
2024-06-16T15:46:49.1797817Z    at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516)
2024-06-16T15:46:49.1799430Z    at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495)
2024-06-16T15:46:49.1814784Z    at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521)
2024-06-16T15:46:49.1816132Z    at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11)
2024-06-16T15:46:49.1817469Z    at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11)
2024-06-16T15:46:49.1818968Z    at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
2024-06-16T15:46:49.1820118Z    at java.base/java.lang.Thread.run(Thread.java:840)

This is due to this piece of code in ClassTransformingBuildStep (usage of cl):

        shutdown.addCloseTask(ClassTransformingBuildStep::reset, true);
        lastTransformers = new BiFunction<String, byte[], byte[]>() {
            @Override
            public byte[] apply(String className, byte[] originalBytes) {

                List<BytecodeTransformerBuildItem> classTransformers = bytecodeTransformers.get(className);
                if (classTransformers == null) {
                    return originalBytes;
                }
                boolean continueOnFailure = classTransformers.stream()
                        .filter(a -> !a.isContinueOnFailure())
                        .findAny().isEmpty();
                List<BiFunction<String, ClassVisitor, ClassVisitor>> visitors = classTransformers.stream()
                        .sorted(Comparator.comparingInt(BytecodeTransformerBuildItem::getPriority))
                        .map(BytecodeTransformerBuildItem::getVisitorFunction)
                        .filter(Objects::nonNull)
                        .collect(Collectors.toList());
                List<BiFunction<String, byte[], byte[]>> preVisitFunctions = classTransformers.stream()
                        .sorted(Comparator.comparingInt(BytecodeTransformerBuildItem::getPriority))
                        .map(BytecodeTransformerBuildItem::getInputTransformer)
                        .filter(Objects::nonNull)
                        .collect(Collectors.toList());
                ClassLoader old = Thread.currentThread().getContextClassLoader();
                try {
                    Thread.currentThread().setContextClassLoader(transformCl);
                    String classFileName = fromClassNameToResourceName(className);
                    List<ClassPathElement> archives = cl.getElementsWithResource(classFileName);
  1. Issues in DevUIGrpcSmokeTest

Hard to figure out what's going wrong here as we don't have the stacktraces handy.

  1. RESTEasy/Config in dev mode

:white_check_mark: This one has been solved by a follow up commit.

``` 2024-06-16T15:34:06.2997052Z 2024-06-16 15:34:05,500 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-1) HTTP Request to /max-body-size failed, error id: a52a04c2-20ea-427c-8809-68804e849435-1 [Error Occurred After Shutdown]: java.lang.ExceptionInInitializerError 2024-06-16T15:34:06.3000384Z at org.jboss.resteasy.core.providerfactory.ResteasyProviderFactoryImpl.registerBuiltin(ResteasyProviderFactoryImpl.java:201) 2024-06-16T15:34:06.3002613Z at org.jboss.resteasy.spi.ResteasyProviderFactory.getInstance(ResteasyProviderFactory.java:115) 2024-06-16T15:34:06.3004440Z at io.quarkus.resteasy.runtime.standalone.RequestDispatcher.service(RequestDispatcher.java:91) 2024-06-16T15:34:06.3006078Z at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler.dispatch(VertxRequestHandler.java:151) 2024-06-16T15:34:06.3007707Z at io.quarkus.resteasy.runtime.standalone.VertxRequestHandler$1.run(VertxRequestHandler.java:97) 2024-06-16T15:34:06.3009191Z at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:599) 2024-06-16T15:34:06.3010602Z at org.jboss.threads.EnhancedQueueExecutor$Task.doRunWith(EnhancedQueueExecutor.java:2516) 2024-06-16T15:34:06.3011953Z at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2495) 2024-06-16T15:34:06.3013305Z at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1521) 2024-06-16T15:34:06.3014580Z at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:11) 2024-06-16T15:34:06.3015847Z at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:11) 2024-06-16T15:34:06.3028336Z at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 2024-06-16T15:34:06.3029607Z at java.base/java.lang.Thread.run(Thread.java:840) 2024-06-16T15:34:06.3031526Z Caused by: java.lang.IllegalStateException: Class loader Quarkus Runtime ClassLoader: DEV for testChunkedLargeRequest() (QuarkusDevModeTest) restart no:0 has been closed and may not be accessed anymore 2024-06-16T15:34:06.3034400Z at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen(QuarkusClassLoader.java:727) 2024-06-16T15:34:06.3036325Z at io.quarkus.bootstrap.classloading.QuarkusClassLoader.getResources(QuarkusClassLoader.java:226) 2024-06-16T15:34:06.3038011Z at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.nextProviderClass(ServiceLoader.java:1203) 2024-06-16T15:34:06.3176043Z at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNextService(ServiceLoader.java:1228) 2024-06-16T15:34:06.3177925Z at java.base/java.util.ServiceLoader$LazyClassPathLookupIterator.hasNext(ServiceLoader.java:1273) 2024-06-16T15:34:06.3179227Z at java.base/java.util.ServiceLoader$2.hasNext(ServiceLoader.java:1309) 2024-06-16T15:34:06.3180271Z at java.base/java.util.ServiceLoader$3.hasNext(ServiceLoader.java:1393) 2024-06-16T15:34:06.3181646Z at io.smallrye.config.SmallRyeConfigProviderResolver.getFactoryFor(SmallRyeConfigProviderResolver.java:102) 2024-06-16T15:34:06.3183298Z at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:78) 2024-06-16T15:34:06.3184881Z at io.smallrye.config.SmallRyeConfigProviderResolver.getConfig(SmallRyeConfigProviderResolver.java:66) 2024-06-16T15:34:06.3186265Z at org.eclipse.microprofile.config.ConfigProvider.getConfig(ConfigProvider.java:85) 2024-06-16T15:34:06.3188279Z at org.jboss.resteasy.microprofile.config.ConfigConfiguration.(ConfigConfiguration.java:36) 2024-06-16T15:34:06.3189993Z at org.jboss.resteasy.microprofile.config.ConfigConfigurationFactory.getConfiguration(ConfigConfigurationFactory.java:32) 2024-06-16T15:34:06.3191645Z at org.jboss.resteasy.plugins.providers.RegisterBuiltin$5.run(RegisterBuiltin.java:178) 2024-06-16T15:34:06.3193059Z at org.jboss.resteasy.plugins.providers.RegisterBuiltin$5.run(RegisterBuiltin.java:175) 2024-06-16T15:34:06.3194411Z at java.base/java.security.AccessController.doPrivileged(AccessController.java:318) 2024-06-16T15:34:06.3195855Z at org.jboss.resteasy.plugins.providers.RegisterBuiltin.isGZipEnabled(RegisterBuiltin.java:175) 2024-06-16T15:34:06.3197355Z at org.jboss.resteasy.plugins.providers.RegisterBuiltin.(RegisterBuiltin.java:37) ```
  1. Wiremock

:white_check_mark: This one has been solved by a follow up commit.

Got it with Stork: ``` 2024-06-16T16:23:09.0013999Z Suppressed: java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for StorkResponseTimeLoadBalancerTest (QuarkusUnitTest) has been closed and may not be accessed anymore 2024-06-16T16:23:09.0016332Z at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen(QuarkusClassLoader.java:727) 2024-06-16T16:23:09.0017924Z at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:503) 2024-06-16T16:23:09.0019360Z at wiremock.org.eclipse.jetty.io.ManagedSelector.doStop(ManagedSelector.java:135) 2024-06-16T16:23:09.0020806Z at wiremock.org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132) 2024-06-16T16:23:09.0022389Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182) 2024-06-16T16:23:09.0023989Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205) 2024-06-16T16:23:09.0025461Z at wiremock.org.eclipse.jetty.io.SelectorManager.doStop(SelectorManager.java:280) 2024-06-16T16:23:09.0026870Z at wiremock.org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132) 2024-06-16T16:23:09.0028472Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182) 2024-06-16T16:23:09.0030097Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205) 2024-06-16T16:23:09.0031630Z at wiremock.org.eclipse.jetty.server.AbstractConnector.doStop(AbstractConnector.java:425) 2024-06-16T16:23:09.0033181Z at wiremock.org.eclipse.jetty.server.AbstractNetworkConnector.doStop(AbstractNetworkConnector.java:82) 2024-06-16T16:23:09.0034923Z at wiremock.org.eclipse.jetty.server.ServerConnector.doStop(ServerConnector.java:240) 2024-06-16T16:23:09.0036442Z at wiremock.org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132) 2024-06-16T16:23:09.0037797Z at wiremock.org.eclipse.jetty.server.Server.doStop(Server.java:506) ``` Also got it with REST Client: ``` 2024-06-16T16:26:18.4058720Z ##[error]Exception in thread "Thread-3842" MultiException[java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore, java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore, java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore, java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore, java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore, java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore, java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore] 2024-06-16T16:26:18.4070116Z at wiremock.org.eclipse.jetty.util.MultiException.ifExceptionThrow(MultiException.java:117) 2024-06-16T16:26:18.4071515Z at wiremock.org.eclipse.jetty.server.Server.doStop(Server.java:531) 2024-06-16T16:26:18.4072926Z at wiremock.org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132) 2024-06-16T16:26:18.4074522Z at com.github.tomakehurst.wiremock.jetty.JettyHttpServer.stop(JettyHttpServer.java:178) 2024-06-16T16:26:18.4075921Z at com.github.tomakehurst.wiremock.WireMockServer.stop(WireMockServer.java:182) 2024-06-16T16:26:18.4077330Z at com.github.tomakehurst.wiremock.WireMockServer.lambda$shutdown$3(WireMockServer.java:215) 2024-06-16T16:26:18.4078476Z at java.base/java.lang.Thread.run(Thread.java:840) 2024-06-16T16:26:18.4080249Z Suppressed: java.lang.IllegalStateException: Class loader Quarkus Base Runtime ClassLoader: TEST for TrustAllTest (QuarkusUnitTest) has been closed and may not be accessed anymore 2024-06-16T16:26:18.4082406Z at io.quarkus.bootstrap.classloading.QuarkusClassLoader.ensureOpen(QuarkusClassLoader.java:727) 2024-06-16T16:26:18.4084348Z at io.quarkus.bootstrap.classloading.QuarkusClassLoader.loadClass(QuarkusClassLoader.java:503) 2024-06-16T16:26:18.4085850Z at wiremock.org.eclipse.jetty.io.ManagedSelector.doStop(ManagedSelector.java:135) 2024-06-16T16:26:18.4087331Z at wiremock.org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132) 2024-06-16T16:26:18.4088957Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182) 2024-06-16T16:26:18.4090689Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205) 2024-06-16T16:26:18.4092202Z at wiremock.org.eclipse.jetty.io.SelectorManager.doStop(SelectorManager.java:280) 2024-06-16T16:26:18.4093675Z at wiremock.org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132) 2024-06-16T16:26:18.4095285Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.stop(ContainerLifeCycle.java:182) 2024-06-16T16:26:18.4096931Z at wiremock.org.eclipse.jetty.util.component.ContainerLifeCycle.doStop(ContainerLifeCycle.java:205) 2024-06-16T16:26:18.4098521Z at wiremock.org.eclipse.jetty.server.AbstractConnector.doStop(AbstractConnector.java:425) 2024-06-16T16:26:18.4100470Z at wiremock.org.eclipse.jetty.server.AbstractNetworkConnector.doStop(AbstractNetworkConnector.java:82) 2024-06-16T16:26:18.4102089Z at wiremock.org.eclipse.jetty.server.ServerConnector.doStop(ServerConnector.java:240) 2024-06-16T16:26:18.4103614Z at wiremock.org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:132) 2024-06-16T16:26:18.4104900Z at wiremock.org.eclipse.jetty.server.Server.doStop(Server.java:506) 2024-06-16T16:26:18.4105674Z ... 5 more ```
quarkus-bot[bot] commented 3 weeks ago

/cc @geoand (kubernetes,minikube), @iocanel (kubernetes)

gsmet commented 3 weeks ago

/cc @dmlloyd @cescoffier I think your feedback will be invaluable here too.

cescoffier commented 3 weeks ago

There is a hack iterating over Vert.x event loops to reset the TCCL in dev mode. That might be the problem.

geoand commented 3 weeks ago

https://github.com/quarkusio/quarkus/pull/40942#issuecomment-2171657096 might also be related

dmlloyd commented 2 weeks ago

We are very cavalier about how we create and use class loaders. Since we don't have any kind of determinism around them, we end up with non-deterministic consequences. Throwing an exception is a good start, but troubleshooting these problems is going to be pretty tricky I suspect.

HerrDerb commented 2 weeks ago

Can this cause classloader issues during QuarkusTest? Since shortly, we experience inconsistent classloader issues when running QuarkusTests within the scope of the actual test class, of which we are still confused.

java.util.concurrent.ExecutionException: java.lang.ClassCastException: class com.company.MyClass cannot be cast to class com.company.MyClass (com.company.MyClass  is in unnamed module of loader 'app'; com.company.MyClass  is in unnamed module of loader io.quarkus.bootstrap.classloading.QuarkusClassLoader @3aeba81)
dmlloyd commented 2 weeks ago

Can this cause classloader issues during QuarkusTest? Since shortly, we experience inconsistent classloader issues when running QuarkusTests within the scope of the actual test class, of which we are still confused.

No, more likely you need #40906 for this one, but it's just a guess based on this small snippet. Once that PR is merged, if you still have an issue, then consider filing a bug or opening a discussion topic for it.