Bug Report: Payara 6.2023.11 - StatelessSessionBean with OpenTelemetry annotation @WithSpan throws jakarta.ejb.CreateException

[pioneer2k]

Brief Summary

We have a StatelessSessionBean in our product that uses a scheduled method togehter with a @WithSpan annotation from the OpenTelemetry package (io.opentelemetry.instrumentation), which is now included in Payara 6.2023.11. With Payara 6.2023.10 and before we had no problems. With Payara 6.2023.11 we receive the following exception everytime the method gets triggered. When we remove the @WithSpan annotation the exception does not occur!

[#|2023-11-21T16:22:15.011+0000|WARNING||jakarta.enterprise.ejb.container|_ThreadID=245;_ThreadName=__ejb-thread-pool12;_TimeMillis=1700583735011;_LevelValue=900;|jakarta.ejb.EJBException: jakarta.ejb.EJBException: jakarta.ejb.CreateException: Could not create stateless EJB
    at com.sun.ejb.containers.StatelessSessionContainer._getContext(StatelessSessionContainer.java:399)
    at com.sun.ejb.containers.BaseContainer.getContext(BaseContainer.java:2607)
    at com.sun.ejb.containers.BaseContainer.preInvoke(BaseContainer.java:2024)
    at com.sun.ejb.containers.BaseContainer.callEJBTimeout(BaseContainer.java:4042)
    at com.sun.ejb.containers.EJBTimerService.deliverTimeout(EJBTimerService.java:1208)
    at com.sun.ejb.containers.EJBTimerService$TaskExpiredWork.run(EJBTimerService.java:1937)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
    at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: jakarta.ejb.EJBException: jakarta.ejb.CreateException: Could not create stateless EJB
    at com.sun.ejb.containers.StatelessSessionContainer$SessionContextFactory.create(StatelessSessionContainer.java:653)
    at com.sun.ejb.containers.util.pool.NonBlockingPool.getObject(NonBlockingPool.java:219)
    at com.sun.ejb.containers.StatelessSessionContainer._getContext(StatelessSessionContainer.java:395)
    ... 10 more
Caused by: jakarta.ejb.CreateException: Could not create stateless EJB
    at com.sun.ejb.containers.StatelessSessionContainer.createStatelessEJB(StatelessSessionContainer.java:473)
    at com.sun.ejb.containers.StatelessSessionContainer$SessionContextFactory.create(StatelessSessionContainer.java:651)
    ... 12 more
Caused by: java.lang.reflect.InvocationTargetException
    at com.sun.ejb.containers.BaseContainer.createEjbInstanceAndContext(BaseContainer.java:1725)
    at com.sun.ejb.containers.StatelessSessionContainer.createStatelessEJB(StatelessSessionContainer.java:419)
    ... 13 more
Caused by: org.jboss.weld.exceptions.WeldException: WELD-001537: An InjectionTarget is created for a class fish.payara.microprofile.telemetry.tracing._WithSpanMethodInterceptorBean_Serializable which does not have any appropriate constructor.
    at org.jboss.weld.util.InjectionTargets.createNonProducibleInjectionTarget(InjectionTargets.java:82)
    at org.jboss.weld.util.InjectionTargets.createNonProducibleInjectionTarget(InjectionTargets.java:48)
    at org.jboss.weld.manager.InjectionTargetFactoryImpl.chooseInjectionTarget(InjectionTargetFactoryImpl.java:126)
    at org.jboss.weld.manager.InjectionTargetFactoryImpl.createInjectionTarget(InjectionTargetFactoryImpl.java:88)
    at org.jboss.weld.bean.ManagedBean.<init>(ManagedBean.java:102)
    at org.jboss.weld.bean.InterceptorImpl.<init>(InterceptorImpl.java:64)
    at org.jboss.weld.bean.InterceptorImpl.of(InterceptorImpl.java:60)
    at org.glassfish.weld.services.JCDIServiceImpl.createInterceptorInstance(JCDIServiceImpl.java:468)
    at com.sun.ejb.containers.BaseContainer.createEjbInterceptors(BaseContainer.java:1800)
    at com.sun.ejb.containers.BaseContainer.createEmptyContextAndInterceptors(BaseContainer.java:1689)
    at com.sun.ejb.containers.BaseContainer.createEjbInstanceAndContext(BaseContainer.java:1703)
    ... 14 more
Caused by: org.jboss.weld.exceptions.DefinitionException: WELD-001537: An InjectionTarget is created for a class fish.payara.microprofile.telemetry.tracing._WithSpanMethodInterceptorBean_Serializable which does not have any appropriate constructor.
    at org.jboss.weld.util.InjectionTargets.createNonProducibleInjectionTarget(InjectionTargets.java:76)

Expected Outcome

No exception should occur.

Current Outcome

see summary

Reproducer

an example of our StatelessSessionBean:

import io.opentelemetry.instrumentation.annotations.WithSpan;
import jakarta.ejb.LocalBean;
import jakarta.ejb.Schedule;
import jakarta.ejb.Stateless;
import jakarta.enterprise.inject.Instance;
import jakarta.inject.Inject;
import lombok.NoArgsConstructor;
import lombok.extern.log4j.Log4j2;

@Stateless
@LocalBean
@NoArgsConstructor
@Log4j2
public class AccountingQueueProcessorBean extends QueueProcessorBase {

    @Inject
    Instance<AccountingQueueWorker> workerFactory;

    @Schedule(second = "*/30", minute = "*", hour = "*", persistent = false)
    @WithSpan
    public void schedule() {
        try {
            super.process(workerFactory);
        } catch (Exception e) {
            log.catching(e);
        }
    }
}

Operating System

Alpine Linux

JDK Version

zulu-openjdk-alpine:17.0.9-jre

Payara Distribution

Payara Micro

[artur-mal]

Hi pioneer2k. I am sorry for not coming back to you earlier.

I managed to reproduce this issue with Payara 6.2023.12.

I have escalated this to the platform development team with internal JIRA FISH-8156

I will let you know once we can proceed with the development of the fix.

Regards, Artur