Closed dan-v-elevate closed 5 years ago
@dan-v-elevate Apologies for not responding sooner.
We have just released a new version 2.0.0 for the Event Hubs library which includes the fix for #186 Please see https://github.com/Azure/azure-event-hubs-node/issues/186#issuecomment-477364700
Do you still see the other errors that you mentioned in the issue description?
The TypeError: Cannot read property 'address' of undefined
should have an accompanying stack trace, which will be helpful for us to figure out what the problem is.
The other errors Error: Detach already received
and Error: transfer after detach
are related to the underlying message transfer details in AMQP which is hard for us to debug without more logs. If you still see this after updating to version 2.0.0 of this library, then I can work with you to enable logging to understand the issue better
@ramya-rao-a - thanks for the update on this! It seems like the majority of our issues are coming from 'A link to connection '.....' $cbs node has already been opened' issue. We will pull in 2.0 and see if that helps.
@ramya-rao-a - just wanted to let you know that 2.0 seems to have fixed our issues. Thanks!
That's great to hear, thanks @dan-v-elevate!
Describe the bug We are using this library 'azure/event-hub' in a v2 Azure function with an event hub trigger. The function essentially reads in batches of data, transforms it, and outputs it to a number of other event hubs. Note that we are using the consumption plan, so we have somewhat limited insight into the underlying hosts where this is running. The issue we are seeing results in the azure function host crashing and not recovering (opened an issue for that first https://github.com/Azure/azure-functions-host/issues/4079) and we get a very limited set of errors bubbled up that appears to possibly be related to this library. The errors:
Error: Detach already received
,Error: transfer after detach
, andTypeError: Cannot read property 'address' of undefined
is what we are seeing in the stack trace which looks pretty similar to another issue already posted here #171 . To be clear, I don't know for certain this is the issue based on the limited stack trace, but if you could provide any input for helping further debug if it is or isn't that would be really appreciated.Stack trace from azure functions
``` [ { "severityLevel":"Error", "outerId":"0", "message":"Exception while executing function: Functions.iot_intake", "type":"Microsoft.Azure.WebJobs.Host.FunctionInvocationException", "parsedStack":[ { "assembly":"System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e", "method":"System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw", "level":0, "line":0 }, { "assembly":"Microsoft.Azure.WebJobs.Host, Version=3.0.4.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35", "method":"Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor+To Reproduce We don't have a simple reproduction of the issue at the moment, but it has happened very regularly for us over the past month. In the Azure function we establish a few connections to different event hubs (eventHubClient.createFromConnectionString) outside of the function handler and then during function executions we are sending data to these event hubs (await client.send({ body: output, partitionKey: partitionKey)). Sometimes after recycling hosts we will hit this issue immediately and other times we can go a few days without hitting it.
Expected behavior Expect connections to event hub to keep being able to send data without throwing these exceptions.
Package-name: azure/event-hubs Package-version: 1.0.8 node.js version: 8.11.1 OS name and version: Running in Azure functions consumption plan on Windows (32 bit)
Additional Information Additionally I'm also seeing the issue #186 fairly frequently and not sure if this is related/contributing to this issue.
Stack trace for InvalidOperationError
``` Timestamp: 2019-02-13T12:22:48.2390823Z Inocation ID: dc2f9399-9f5e-4a61-bc1a-23d5f7d21ccd { InvalidOperationError: Error sending batch to rules event hub -- A link to connection '237673' $cbs node has already been opened. at Object.translate (D:\home\site\wwwroot\node_modules\@azure\amqp-common\dist\lib\errors.js:527:17) at CbsClient.