Occasional eventhub sending message error

[izvictor]

Hello, We have noticed an occasional error during sending an eventhub message :

"InnerException":{"IsTransient":false,"Reason":1,"EventHubName":"eventhub-pi-tepqa","Message":"Unable to create the items needed to communicate with the Event Hubs service. (eventhub-pi-tepqa)","Data":{},"InnerException":{"ClassName":"System.ObjectDisposedException","Message":"Cannot access a disposed object.","Data":null,"InnerException":null,"HelpURL":null,"StackTraceString":"   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.Amqp.AsyncResult.End[TAsyncResult](IAsyncResult result)
   at Microsoft.Azure.Amqp.AmqpObject.OpenAsyncResult.End(IAsyncResult result)
   at Microsoft.Azure.Amqp.AmqpObject.EndOpen(IAsyncResult result)
   at Microsoft.Azure.Amqp.Transport.AmqpTransportInitiator.HandleTransportOpened(IAsyncResult result)
   at Microsoft.Azure.Amqp.Transport.AmqpTransportInitiator.OnTransportOpenCompete(IAsyncResult result)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.EventHubs.Amqp.AmqpConnectionScope.<CreateAndOpenConnectionAsync>d__66.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Amqp.FaultTolerantAmqpObject`1.<OnCreateAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.Amqp.Singleton`1.<GetOrCreateAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.Amqp.Singleton`1.<GetOrCreateAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.EventHubs.Amqp.AmqpConnectionScope.<OpenProducerLinkAsync>d__64.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.EventHubs.Amqp.AmqpProducer.<CreateLinkAndEnsureProducerStateAsync>d__43.MoveNext()","RemoteStackTraceString":null,"RemoteStackIndex":0,"ExceptionMethod":"8
Throw
mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
System.Runtime.ExceptionServices.ExceptionDispatchInfo
Void Throw()","HResult":-2146232798,"Source":"mscorlib","WatsonBuckets":null,"ObjectName":"sasl2715"},"StackTrace":"   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.EventHubs.Amqp.AmqpProducer.<CreateLinkAndEnsureProducerStateAsync>d__43.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Azure.Amqp.FaultTolerantAmqpObject`1.<OnCreateAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.Amqp.Singleton`1.<GetOrCreateAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.Azure.Amqp.Singleton`1.<GetOrCreateAsync>d__13.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.EventHubs.Amqp.AmqpProducer.<SendAsync>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.EventHubs.Amqp.AmqpProducer.<SendAsync>d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.EventHubs.Amqp.AmqpProducer.<SendAsync>d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.EventHubs.Producer.EventHubProducerClient.<SendInternalAsync>d__53.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Azure.Messaging.EventHubs.Producer.EventHubProducerClient.<SendInternalAsync>d__53.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.EventHubs.Producer.EventHubProducerClient.<SendAsync>d__44.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Azure.Messaging.EventHubs.Producer.EventHubProducerClient.<SendAsync>d__43.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at PI2Cloud.Services.DataPipes.DataReceivers.PIDataReceiver.<OnNext>d__4.MoveNext()","HelpLink":null,"Source":"mscorlib","HResult":-2146233088},"StackTrace":null,"HelpLink":null,"Source":null,"HResult":-2146233088}|

The error disappears after a certain amount of time. Is this some kind of transient error ? the event hub producer configuration are the following:

_eventHubClient = new EventHubProducerClient(target.ConnectionString, target.EventHubName, new EventHubProducerClientOptions()
                    {
                        RetryOptions = new EventHubsRetryOptions
                        {
                            Mode = EventHubsRetryMode.Exponential,
                            MaximumRetries = 10,
                            Delay = TimeSpan.FromSeconds(1),
                            MaximumDelay = TimeSpan.FromMinutes(1)
                        },
                        ConnectionOptions = new EventHubConnectionOptions
                        {
                            TransportType = EventHubsTransportType.AmqpWebSockets
                        }
                    });

Thanks for help. Best regards.

[xinchen10]

Seems that you are using the newest Event Hubs SDK. For the Producer this error is transient. The TCP socket was reset while the SDK is trying to create a communication channel. However, you should not get the ObjectDisposedException directly since you are using the EH SDK, which is supposed to handle this error and convert to a transient exception type of the SDK. If a retry policy is defined, the SDK should retry on it if there is time left. You can reach out to the SDK team for more details.