logdna / logger-node

A nodejs logger client for LogDNA
MIT License
34 stars 17 forks source link

A connection-based error occurred that will not be retried #79

Closed rashadksh closed 1 year ago

rashadksh commented 1 year ago

Suddenly my docker stopped and threw this error.

[1] Error: A connection-based error occurred that will not be retried. See meta data for details.
[1]     at /app/backend/node_modules/@logdna/logger/lib/logger.js:867:29
[1]     at processTicksAndRejections (internal/process/task_queues.js:77:11)
[1] 
[1] npm ERR! code ELIFECYCLE
[1] npm ERR! errno 1
[1] npm ERR! haystack@0.4.248 server:production: `node --max_old_space_size=15000 ./dist/src/server.js`
[1] npm ERR! Exit status 1
[1] npm ERR! 
[1] npm ERR! Failed at the haystack@0.4.248 server:production script.
[1] npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
[1] 
npm ERR! A complete log of this run can be found in:
[1] npm ERR!     /root/.npm/_logs/2023-05-18T14_45_51_040Z-debug.log
[1] npm run server:production exited with code 1
darinspivey commented 1 year ago

Hi, thanks for sharing this. The ELIFECYCLE error is just because the running of the server is wrapped in npm, and it's saying "something else failed." In this case, it looks like communicating with the ingestion service returned an error that could not be gracefully handled (and retried).

What the error is telling the user is that additional information can be found in the meta property of the error that's emitted, so that's what needs inspected. From there, we may have some more answers. Until then, make sure you have an error handler and that your dumping the full information to the logs. It's unclear if npm is cutting anything off, or if the meta property was not serialized as output.

Can you provide more information as to what your code and error handling looks like? In the "Best Practices" section of the README, there's a bit on error handling.

rashadksh commented 1 year ago

Thanks for the reply, we are actually using logdna-winston like this:

const consoleTransport = new transports.Console({
  format: format.combine(
    format.colorize(),
    format.printf(info => `${info.timestamp} ${info.level} [${info.label}]: ${info.message}`),
  ),
});

const logdnaTransport = new LogdnaWinston({
  key: '123',
  app: 'Backend',
  env: process.env.NODE_ENV,
  level: 'debug',  
});

export const logger = createLogger({
  level: 'http',
  format: format.combine(format.label({ label: path.basename(label) }), format.timestamp({ format: 'YYYY-MM-DD HH:mm:ss' })),
  transports: [consoleTransport, logdnaTransport],
  exitOnError: false,
});

will adding logger.on('error') work here? since logger now is an instance of Winston, not LogDNA logger

rashadksh commented 1 year ago

@darinspivey have you had a chance to look at this?

Thanks in advance

darinspivey commented 1 year ago

Yes you can. The logger-node instance is exposed through the logdna-winston transport. You can attach a listener to that. In this example, providing a bad key will cause the error you experienced, and show the meta details you need:

const winston = require('winston')
const logdnaTransport = require('logdna-winston')

const key = 'BAD KEY'

const transport = new logdnaTransport({
  key
, maxLevel: 'verbose'
})

transport.logger.on('error', (err) => {
  console.error('There was an error', err)
})

const logger = winston.createLogger({
  transports:[transport]
})
logger.silly('This will not be logged')
logger.verbose('This will be logged')

Yields the following result, which indicates a 403 auth error due to the bad key:

There was an error Error: A connection-based error occurred that will not be retried. See meta data for details.
    at [snip]/node_modules/@logdna/logger/lib/logger.js:811:25
    at processTicksAndRejections (node:internal/process/task_queues:78:11) {
  meta: {
    actual: 'Request failed with status code 403',
    code: 403,
    firstLine: 'This will be logged',
    lastLine: null,
    retrying: false,
    attempts: 1,
    headers: {
      'Content-Type': 'application/json; charset=UTF-8',
      'user-agent': '@logdna/logger/2.4.0 (logdna-winston/4.0.1)'
    },
    url: "https://logs.logdna.com/logs/ingest?now=1684933814514&hostname=Darin's-MacBook-Pro&mac=&ip=&tags="
  }
darinspivey commented 1 year ago

Closing. No action necessary after clarifying usage with the user.