[BUG] EventHubConsumerAsyncClient - Connection Aborted

[alemag1986]

Describe the bug Using Event Hub PREMIUM tier, trying to connect from a spring boot application running on a container in a webapp getting connection aborted, and no description of the error:

2024-05-10T05:25:48.547189346Z reactor.core.Exceptions$ErrorCallbackNotImplemented: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A] 2024-05-10T05:25:48.547193246Z Caused by: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A]

Exception or Stack Trace 24-05-10T05:25:48.547189346Z reactor.core.Exceptions$ErrorCallbackNotImplemented: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A] 2024-05-10T05:25:48.547193246Z Caused by: com.azure.core.amqp.exception.AmqpException: org.apache.qpid.proton.engine.TransportException: connection aborted, errorContext[NAMESPACE: ######.servicebus.windows.net. ERROR CONTEXT: N/A] 2024-05-10T05:25:48.547197446Z at com.azure.core.amqp.implementation.ExceptionUtil.toException(ExceptionUtil.java:90) ~[azure-core-amqp-2.9.3.jar!/:2.9.3] 2024-05-10T05:25:48.547201746Z at com.azure.core.amqp.implementation.handler.ConnectionHandler.notifyErrorContext(ConnectionHandler.java:363) ~[azure-core-amqp-2.9.3.jar!/:2.9.3] 2024-05-10T05:25:48.547205646Z at com.azure.core.amqp.implementation.handler.ConnectionHandler.onTransportError(ConnectionHandler.java:259) ~[azure-core-amqp-2.9.3.jar!/:2.9.3] 2024-05-10T05:25:48.547209546Z at org.apache.qpid.proton.engine.BaseHandler.handle(BaseHandler.java:191) ~[proton-j-0.34.1.jar!/:na] 2024-05-10T05:25:48.547213446Z at org.apache.qpid.proton.engine.impl.EventImpl.dispatch(EventImpl.java:108) ~[proton-j-0.34.1.jar!/:na] 2024-05-10T05:25:48.547218246Z at org.apache.qpid.proton.reactor.impl.ReactorImpl.dispatch(ReactorImpl.java:324) ~[proton-j-0.34.1.jar!/:na] 2024-05-10T05:25:48.547221846Z at org.apache.qpid.proton.reactor.impl.ReactorImpl.process(ReactorImpl.java:291) ~[proton-j-0.34.1.jar!/:na] 2024-05-10T05:25:48.547225446Z at com.azure.core.amqp.implementation.ReactorExecutor.run(ReactorExecutor.java:91) ~[azure-core-amqp-2.9.3.jar!/:2.9.3] 2024-05-10T05:25:48.547228746Z at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) ~[reactor-core-3.6.3.jar!/:3.6.3] 2024-05-10T05:25:48.547232346Z at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) ~[reactor-core-3.6.3.jar!/:3.6.3] 2024-05-10T05:25:48.547236346Z at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[na:na] 2024-05-10T05:25:48.547251746Z at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source) ~[na:na] 2024-05-10T05:25:48.547255446Z at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na] 2024-05-10T05:25:48.547258946Z at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na] 2024-05-10T05:25:48.547262746Z at java.base/java.lang.Thread.run(Unknown Source) ~[na:na] 2024-05-10T05:25:48.547266547Z 2024-05-10T05:25:48.547269647Z 2024-05-10T05:25:48.542Z ERROR 15 --- [tor-executor-16] c.a.core.amqp.implementation.RetryUtil : Time out creating management node. 2024-05-10T05:25:48.547273647Z Retries exhausted: 3/3

Code Snippet

public EventHubConsumerAsyncClient eventHubConsumerAsyncClient() {

    EventHubClientBuilder eventHubClientBuilder = new EventHubClientBuilder()
        .connectionString(namespace, eventHubName)
        .consumerGroup(consumerGroupName);

    EventHubConsumerAsyncClient client = eventHubClientBuilder.buildAsyncConsumerClient();
    // Perform a health check to ensure the connection is successful
    try {
        client.receive().blockFirst();
        log.info("Connected to Event Hub successfully.");
        isConnected = true;
    } catch (Exception e) {
        log.error("Error connecting to Event Hub: {}", e.getMessage());
        log.error("Error StackTrace: {}", e.toString());
    }
    return client;
}

This is a health check basically. This same code, with same web app configuration is working on our Development environment that is NOT using EventHub PREMIUM tier.

Expected behavior Connection to Event Hub and start to recieve messages.

Setup (please complete the following information): Library versions:

implementation 'com.azure:azure-messaging-eventhubs:5.18.3' implementation 'com.azure:azure-identity:1.12.1'

Docker Base: FROM amazoncorretto:21-alpine as corretto-jdk

Java: plugins { id 'java' id 'org.springframework.boot' version '3.2.3' id 'io.spring.dependency-management' version '1.1.4' id 'org.sonarqube' version '4.4.1.3373' }

java { sourceCompatibility = '21' }

Information Checklist Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [X] Bug Description Added
• [X] Repro Steps Added
• [X] Setup information Added

[github-actions[bot]]

@anuchandy @conniey @lmolkova

[github-actions[bot]]

Thank you for your feedback. Tagging and routing to the team member best able to assist.

[alemag1986]

Hi there, We are still troubleshooting the issue from our side... it looks like the issue happens when the docker image is build using Ubuntu. Our docker build process happens in Azure DevOps Pipeline Ubuntu latest: https://github.com/actions/runner-images/blob/main/images/ubuntu/Ubuntu2204-Readme.md

[anuchandy]

@alemag1986, are you saying this connection-aborted only occurs with a specific version of Ubuntu and it's not related to the Event Hub tier (standard, premium)? Is this something we reproduce when running on local docker or only on AKS?

[alemag1986]

@anuchandy

• its not happening in Event Hub standard tier.
• it only happens in premium.
• it is reproducible from local docker not need to cloud infrastructure (in the morning will share docker info).

[alemag1986]

The issue is related to Docker Image. The base ubuntu has been tailored for our use case and unfortunately something is missing to support eventhub premium.

Here are the docker file details in case somebody is struggling or want to find what is missing.

# Use the official OpenJDK image as the base image
FROM amazoncorretto:21-alpine as corretto-jdk
WORKDIR /app
COPY ./deps.info /app/deps.info

# required for strip-debug to work
RUN apk add --no-cache binutils

# Build small JRE image
RUN jlink \
         --verbose \
         --add-modules $(cat deps.info) \
         --strip-debug \
         --no-man-pages \
         --no-header-files \
         --compress=2 \
         --output /jre

FROM alpine:3.19

ENV JAVA_HOME=/jre
ENV PATH="${JAVA_HOME}/bin:${PATH}"

COPY --from=corretto-jdk /jre $JAVA_HOME

CMD ["/bin/sh"]

Going to close the issue... since is not related to the library. Most likely, we are missing a requirement.