[Beta] Application crashes if `applicationinsights.azure.com` can't be reached

[marcus13371337]

Hello it seems as if the server can't reach applicationinsights.azure.com when exporting spans the application crashes. I would assume that's not the desired behavior as telemetry is a nice-to-have and not an application crucial feature.

Here is an example log from when that happened!

2023-03-22T20:38:17.974032471Z ApplicationInsights:Envelopes could not be exported and are not retriable. Error message: [
2023-03-22T20:38:17.974110773Z   'getaddrinfo ENOTFOUND northeurope-2.in.applicationinsights.azure.com'
2023-03-22T20:38:17.974118673Z ]
2023-03-22T20:38:17.976989330Z ApplicationInsights:{"stack":"RestError: getaddrinfo ENOTFOUND northeurope-2.in.applicationinsights.azure.com\n    at ClientRequest.<anonymous> (/node_modules/.pnpm/@azure+core-rest-pipeline@1.10.1/node_modules/@azure/core-rest-pipeline/dist/index.js:1700:24)\n    at /node_modules/.pnpm/@opentelemetry+context-async-hooks@1.9.1_@opentelemetry+api@1.4.1/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:55\n    at AsyncLocalStorage.run (node:async_hooks:330:14)\n    at AsyncLocalStorageContextManager.with (/node_modules/.pnpm/@opentelemetry+context-async-hooks@1.9.1_@opentelemetry+api@1.4.1/node_modules/@opentelemetry/context-async-hooks/build/src/AsyncLocalStorageContextManager.js:33:40)\n    at ClientRequest.contextWrapper (/node_modules/.pnpm/@opentelemetry+context-async-hooks@1.9.1_@opentelemetry+api@1.4.1/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:32)\n    at Object.onceWrapper (node:events:628:26)\n    at ClientRequest.emit (node:events:513:28)\n    at TLSSocket.socketErrorListener (node:_http_client:494:9)\n    at TLSSocket.emit (node:events:513:28)\n    at emitErrorNT (node:internal/streams/destroy:157:8)","message":"getaddrinfo ENOTFOUND northeurope-2.in.applicationinsights.azure.com","name":"RestError","code":"ENOTFOUND","request":"[object Object]"} []

2023-03-22T20:38:26.690944014Z ApplicationInsights:Envelopes could not be exported and are not retriable. Error message: [
2023-03-22T20:38:26.691028115Z   'getaddrinfo ENOTFOUND northeurope-2.in.applicationinsights.azure.com'
2023-03-22T20:38:26.691035415Z ]
2023-03-22T20:38:26.701496408Z /node_modules/.pnpm/@azure+core-rest-pipeline@1.10.1/node_modules/@azure/core-rest-pipeline/dist/index.js:1700
2023-03-22T20:38:26.701530309Z                 reject(new RestError(err.message, { code: (_a = err.code) !== null && _a !== void 0 ? _a : RestError.REQUEST_SEND_ERROR, request }));
2023-03-22T20:38:26.701537509Z                        ^
2023-03-22T20:38:26.701541909Z 
2023-03-22T20:38:26.701555409Z RestError: getaddrinfo ENOTFOUND northeurope-2.in.applicationinsights.azure.com
2023-03-22T20:38:26.701560709Z     at ClientRequest.<anonymous> (/node_modules/.pnpm/@azure+core-rest-pipeline@1.10.1/node_modules/@azure/core-rest-pipeline/dist/index.js:1700:24)
2023-03-22T20:38:26.701565309Z     at /node_modules/.pnpm/@opentelemetry+context-async-hooks@1.9.1_@opentelemetry+api@1.4.1/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:55
2023-03-22T20:38:26.701569809Z     at AsyncLocalStorage.run (node:async_hooks:330:14)
2023-03-22T20:38:26.701573810Z     at AsyncLocalStorageContextManager.with (/node_modules/.pnpm/@opentelemetry+context-async-hooks@1.9.1_@opentelemetry+api@1.4.1/node_modules/@opentelemetry/context-async-hooks/build/src/AsyncLocalStorageContextManager.js:33:40)
2023-03-22T20:38:26.701578310Z     at ClientRequest.contextWrapper (/node_modules/.pnpm/@opentelemetry+context-async-hooks@1.9.1_@opentelemetry+api@1.4.1/node_modules/@opentelemetry/context-async-hooks/build/src/AbstractAsyncHooksContextManager.js:50:32)
2023-03-22T20:38:26.701582610Z     at Object.onceWrapper (node:events:628:26)
2023-03-22T20:38:26.701586510Z     at ClientRequest.emit (node:events:513:28)
2023-03-22T20:38:26.701590210Z     at TLSSocket.socketErrorListener (node:_http_client:494:9)
2023-03-22T20:38:26.701594010Z     at TLSSocket.emit (node:events:513:28)
2023-03-22T20:38:26.701597810Z     at emitErrorNT (node:internal/streams/destroy:157:8) {
2023-03-22T20:38:26.701601710Z   code: 'ENOTFOUND',
2023-03-22T20:38:26.701605410Z   statusCode: undefined,
2023-03-22T20:38:26.701609110Z   request: PipelineRequestImpl {
...
2023-03-22T20:38:26.701764113Z     timeout: 0,
2023-03-22T20:38:26.701767813Z     formData: undefined,
2023-03-22T20:38:26.701771613Z     disableKeepAlive: false,
2023-03-22T20:38:26.701775413Z     proxySettings: undefined,
2023-03-22T20:38:26.701779213Z     streamResponseStatusCodes: Set(0) {},
2023-03-22T20:38:26.701782913Z     withCredentials: false,
2023-03-22T20:38:26.701786613Z     abortSignal: undefined,
2023-03-22T20:38:26.701790414Z     tracingOptions: undefined,
2023-03-22T20:38:26.701794114Z     onUploadProgress: undefined,
2023-03-22T20:38:26.701797914Z     onDownloadProgress: undefined,
2023-03-22T20:38:26.701805614Z     requestId: 'b0c23557-0886-4ad9-acfd-087adef6c7d6',
2023-03-22T20:38:26.701809814Z     allowInsecureConnection: false,
2023-03-22T20:38:26.701813614Z     enableBrowserStreams: false
2023-03-22T20:38:26.701817314Z   },
2023-03-22T20:38:26.701821114Z   response: undefined
2023-03-22T20:38:26.701824914Z }
2023-03-22T20:38:26.750497011Z  ELIFECYCLE  Command failed with exit code 1.
2023-03-22T20:38:26.799116606Z  ELIFECYCLE  Command failed with exit code 1.

[hectorhdzg]

@marcus13371337 do you have any more details in the exception?, telemetry failed to be delivered to ingestion endpoint is quite common, I cannot see a single place where this could throw an unhandled exception, can you double check if there is no other exception part of this?

[marcus13371337]

@hectorhdzg I might have found something, we are using node v16, and an unhandled promise rejection in node >=15 will kill the application if I understand it correctly.

From what I can see, the promise created here does not have any .catch-attached as far as I can see.

From the log message I sent the code seems to have passed this line of code which rejects the promise created here, and boom we have an uncaught rejected promise!

[hectorhdzg]

Fixed in latest release

[marcus13371337]

Nice! Thanks for fixing the issue!