Transport does not reliably work with process manager

[mattberther]

The transport does not work reliably when using a process manager (e.g., pm2). In most cases, you see that the rotated log files are either not removed properly, or that the process crashes upon the removal.

Please investigate how to synchronize changes when this transport is run under a process manager.

[parthgoyal]

Hi Matt,

Can we change the number of maxFiles (DailyRotateFile) at runtime?

I created the winston logger as follows:

const { createLogger, format, transports } = require('winston'); const { combine, timestamp, label, printf, splat } = format; require('winston-daily-rotate-file');

const myFormat = printf(({ level, message, label, timestamp }) => { return ${timestamp} [${label}] ${level}: ${message}; });

var transport = new (transports.DailyRotateFile)({ filename: path.join(PATH , 'log_%DATE%.log'), datePattern: 'YYYYMMDD', zippedArchive: false, maxFiles: '7' });

const logger = createLogger({ level: "debug", format: combine( label({ label: process.pid }), timestamp({format: 'YYYY-MM-DD HH:mm:ss'}), splat(), myFormat ), transports: [ transport ] });

Now I am trying to change the maxFiles as follows:

logger.transports[0].options.maxFiles=

But this is not working. Please suggest the way to do the same. Thanks

[mattberther]

@parthgoyal This seems to be a separate issue from the one describe in the OP. Would you please start a new issue for this, as this functionality currently does not exist?

[parthgoyal]

Thanks Matt for prompt reply. I have opened the issue for the same.

[nithil]

Any update on this? we are getting a lot of errors due to this fs.js:1061 - Object.fs.unlinkSync. The node process is crashing & pm2 starts that process again.

[SunshowerC]

dig out something. package: https://github.com/rogerc/file-stream-rotator file: https://github.com/rogerc/file-stream-rotator/edit/master/FileStreamRotator.js line: 616

616: rotateStream.write(str, encoding); // it may write after stream destroy 617: // Handle length of double-byte characters 618: curSize += Buffer.byteLength(str, encoding);

workaound:

// should not write when rotateStream has been destroyed
if(!rotateStream.destroyed) {
    rotateStream.write(str, encoding);
}

at lease works for me.

[mattberther]

Thank you, @SunshowerC. I'm tracking the upstream PR and will incorporate it after it's merged.

[f1mp3r]

Any updates on this?

[mattberther]

@f1mp3r The upstream PR is still open. So, unfortunately, not.

[amitpadova]

Hi Matt, Is there any timeframe to merge the fix ?

[mattberther]

@amitpadova I'm still tracking the upstream PR and will incorporate after it's merged. I cannot comment on timeframes for projects that I am not a maintainer on.

[rogerc]

@mattberther this seems to be an issue with process specific logging rather than routing all logging through the parent process. Just silencing the error is not really a good fix as the PR submitted would just silence the error discarding the data that should be logged and not deal with the root cause.

[techDPM]

Bug Fix for ENOENT Error with Winston DailyRotateFile and PM2

Encountered an ENOENT (no such file or directory) error during log rotation with Winston's DailyRotateFile transport when used alongside PM2. This issue arises because PM2 alters log files in ways Winston doesn't expect, causing attempts to delete non-existent log files (e.g., logs/application-DATE.log).

Solution:

1. Disable PM2 Log Handling: Prevent PM2 from managing log files by setting both out_file and error_file to /dev/null in the PM2 configuration:

   apps: [{
    // other configuration lines
    out_file: "/dev/null",
    error_file: "/dev/null"
   }]

2. Graceful Shutdown on Error: Ensure the application exits gracefully upon unhandled exceptions, after logging the error:

   process.on('uncaughtException', (err) => {
    logger.error('Unhandled Exception', { error: err });
    process.exit(1);
   });

3. Alternative Logging Solutions: Consider using Winston plugins for integrated logging and monitoring, or employ system-level log rotation through tools like logrotate for more flexibility.

This approach mitigates conflicts between PM2's log management and Winston's log rotation, ensuring stable application logging.

[Apollon77]

@techDPM hm ... a more detailed error info and the version you used would be great because the latest updates should have fixed several ENOENT cases ... if there is on "Uncatchaeble" more one we can fix, IF we would know the details.