microsoft / ApplicationInsights-node.js

Microsoft Application Insights SDK for Node.js
MIT License
325 stars 140 forks source link

Events stuck in temp folder #605

Open irriss opened 4 years ago

irriss commented 4 years ago

Hi there,

I am using ApplicationInsights-node with iisnode and noticed that some *.ai files stuck in TEMP\appInsights-key folder forever and nothing is written in stderr or stdout.

The last event in *.ai file is always this exception:

{
   "ver":1,
   "sampleRate":100,
   "tags":{
      "ai.application.ver":"0.1.0",
      "ai.device.id":"",
      "ai.cloud.roleInstance":"ns547004",
      "ai.device.osVersion":"Windows_NT 10.0.14393",
      "ai.cloud.role":"Web",
      "ai.device.osArchitecture":"x64",
      "ai.device.osPlatform":"win32",
      "ai.internal.sdkVersion":"node:1.7.2"
   },
   "data":{
      "baseType":"ExceptionData",
      "baseData":{
         "ver":2,
         "exceptions":[
            {
               "hasFullStack":true,
               "parsedStack":[
                  {
                     "sizeInBytes":359,
                     "level":0,
                     "method":"NodeClient.TelemetryClient.trackException",
                     "assembly":"at NodeClient.TelemetryClient.trackException (C:\\inetpub\\wwwroot\\sitename\\node_modules\\applicationinsights\\out\\Library\\TelemetryClient.js:60:35)",
                     "fileName":"C:\\inetpub\\wwwroot\\sitename\\node_modules\\applicationinsights\\out\\Library\\TelemetryClient.js",
                     "line":60
                  },
                  {
                     "sizeInBytes":337,
                     "level":1,
                     "method":"AutoCollectExceptions.handle",
                     "assembly":"at AutoCollectExceptions.handle (C:\\inetpub\\wwwroot\\sitename\\node_modules\\applicationinsights\\out\\AutoCollection\\Exceptions.js:26:35)",
                     "fileName":"C:\\inetpub\\wwwroot\\sitename\\node_modules\\applicationinsights\\out\\AutoCollection\\Exceptions.js",
                     "line":26
                  },
                  {
                     "sizeInBytes":117,
                     "level":2,
                     "method":"process.emit",
                     "assembly":"at process.emit (events.js:198:13)",
                     "fileName":"events.js",
                     "line":198
                  },
                  {
                     "sizeInBytes":261,
                     "level":3,
                     "method":"processEmit [as emit]",
                     "assembly":"at processEmit [as emit] (C:\\inetpub\\wwwroot\\sitename\\node_modules\\signal-exit\\index.js:155:32)",
                     "fileName":"C:\\inetpub\\wwwroot\\sitename\\node_modules\\signal-exit\\index.js",
                     "line":155
                  },
                  {
                     "sizeInBytes":187,
                     "level":4,
                     "method":"emitPromiseRejectionWarnings",
                     "assembly":"at emitPromiseRejectionWarnings (internal/process/promises.js:119:20)",
                     "fileName":"internal/process/promises.js",
                     "line":119
                  },
                  {
                     "sizeInBytes":173,
                     "level":5,
                     "method":"process._tickCallback",
                     "assembly":"at process._tickCallback (internal/process/next_tick.js:69:34)",
                     "fileName":"internal/process/next_tick.js",
                     "line":69
                  }
               ],
               "message":"[object Promise]",
               "typeName":"Error"
            }
         ],
         "properties":{

         },
         "severityLevel":3
      }
   },
   "iKey":"ai-key",
   "name":"Microsoft.ApplicationInsights.aikey.Exception",
   "time":"2020-03-17T02:41:08.238Z"
}

I have a staging environment with exacty the same code and same exception is sent well to another instance of AI (with different key).

Please help figure out what could be the problem.

irriss commented 4 years ago

Actually, this morning I found even more files stuck in that folder. Not all of them contain that exception.

Is there a way to get additional logs/output to figure out why the files were not processed/deleted?

irriss commented 4 years ago

Now I have many errors like this

ApplicationInsights:Sender [ 'Ingestion endpoint could not be reached 55 consecutive times. There may be resulting telemetry loss. Most recent error:',
  { Error: getaddrinfo ENOTFOUND dc.services.visualstudio.com dc.services.visualstudio.com:443
      at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:56:26)
    errno: 'ENOTFOUND',
    code: 'ENOTFOUND',
    syscall: 'getaddrinfo',
    hostname: 'dc.services.visualstudio.com',
    host: 'dc.services.visualstudio.com',
    port: 443 } ]

But only in one instance of IIS app. Another instance has same app code the only difference is AppInsight key and it do not have any issues with sending events.