Wait with creating files until a log is written

[AlexDM0]

Hi, I'm setting up a logger for my project. This project consists of a number of libraries and a main project using those. I'm in the process of converting the existing logs in all libraries and modules to winston logs.

I like the ability to choose dynamically or via environmental variables whether logging to file is active. Silent alone is appearently not enough, because when the transport is constructed, an audit file and empty log file is created.

This would be annoying for users of the libraries, so now I have to only create the file logger once I want to use it. Would it make sense to only create files when something is actually logged to them?

[mattberther]

@AlexDM0 To make sure I understand what you're requesting, are you suggesting to move the initialization of the stream that starts at https://github.com/winstonjs/winston-daily-rotate-file/blob/98dd8e76f9a6000854bca533c0244cf7bd3dc900/daily-rotate-file.js#L80 and initialize it upon the first use of log()? For example:

DailyRotateFile.prototype.log = function (info, callback) {
    callback = callback || noop;
    this.logStream = this.logStream || createNewStream();

    this.logStream.write(info[MESSAGE] + this.options.eol);
    ...
}

[leiming]

move the initialization of the stream that starts and initialize it upon the first use of log()?

@mattberther That's what I mean if you would support it. Thank you very much.

[AlexDM0]

Somehow I missed the initial reply to my issue, apologies. Without diving into the Winston source, im not sure if that would address the issue. However, if the end result would be that you enable logging to file and that no files are created until a string is actually logged to file then yes :).

[hugo-daclon]

Hi, Is there any updates regarding this feat request ? Would there be a temporary workaround such as overriding a method with a subclass ? something like what chatGPT proposed (does not work):

class CustomDailyRotateFile extends DailyRotateFile {
  log(info, callback) {
    // Check if logs have been written for the current day
    const today = new Date().toISOString().slice(0, 10); // Get the current date in "YYYY-MM-DD" format

    if (this.currentDate !== today) {
      this.openTheStream();
    }

    super.log(info, callback);
  }
}

What I specifficaly want is a bit different, I want to rotate my log files not based on a date but on an argument. like this.

export function createFooLogger(jobName: string): Logger {
    return winston.createLogger({
        level: logLevel,
        levels: winston.config.syslog.levels,
        transports: [
            new winston.transports.DailyRotateFile({
                level: 'info',
                filename: `logs/foo${jobName}.log`,
                zippedArchive: true,
                maxFiles: '30d',
            }),
        ],
    })
}

but it does not work😢

EDIT 1.

As my problem was a bit different and didn't require this transport and I implemented the following solution and I ended up using the winston's default File transport in a function and creating a script that run everyday with node-schedule and deletes files that are older than 1 month.

// Services/Loggers.ts
// ...
function createCustomLogger(label: string, fileName: string) {
    return createLogger({
        levels: config.syslog.levels,
        format: commonCustomFormat
        transports: [
            new transports.File({
                level: 'info',
                filename: path.join(process.cwd(), './logs/', fileName),
                format: fileSpecificCustomFormat
                ),
                zippedArchive: true,
                maxFiles: 30,
                maxsize: 1000000,
                tailable: true,
            }),
            new transports.Console({
                level: consoleLevel,
                stderrLevels: ['crit', 'alert', 'emerg'],
                consoleWarnLevels: ['warning', 'error'],
                format: consoleSpecificCustomFormat
            }),
        ],
    })
}
// ...
export const createPOLogger = (env: Environment, codeCommande: string) =>
    createCustomLogger(`${env.toUpperCase()} PO_${codeCommande}`, path.join(process.env?.PO_LOGS_DIR ?? '', `${codeCommande}.log`)))

//tasks/DeleteOldPOLogs.ts
//...
const timeLimitInDays: number = parseInt(process.env.PO_LOGS_TIME_LIMIT_DAYS ?? '31')
const logDirectory: string = path.join(process.cwd(), './logs', process.env?.PO_LOGS_DIR ?? '')
const statAsync = promisify(fs.stat)
const readdirAsync = promisify(fs.readdir)
const unlinkAsync = promisify(fs.unlink)
const script = async () => {
    const today = moment()
    const directory = await readdirAsync(logDirectory, { withFileTypes: true })
    for (const file of directory) {
        if (!file.isFile()) return
        const filePath = join(logDirectory, file.name)

        const stats = await statAsync(filePath)
        const fileDate = moment(stats.mtime)
        if (moment.duration(today.diff(fileDate)).as('d') >= timeLimitInDays) {
            unlinkAsync(filePath)
                .then(() => logger.info('Product Obligation log deleted.', { file }))
        }
    }
}
// ...

I'm using the same logger config for other purpose (which is why I have config in

[hakunamatata97k]

@mattberther any update on this issue? I'm facing a somewhat similar issue in #378 @AlexDM0 what solution did you end up doing?

[hugo-daclon]

There is an option in winston called lazy which works (not documented in this transport's documentation, but if you're using TS. They have an issue they fixed but didn't release where this lazy option wasn't added to the types. It work in JS though.

I didn't check if it trully works but as they are using the file transport as a base it should.

[hakunamatata97k]

Thanks a lot for mentioning this :+1: i will take a look at it after i get back from gym. i will let you know. :)

[hakunamatata97k]

@Nol-go I modified the code provided in the #378 to make it include the lazy option, but it still produces the same behaviour any idea? It creates the audit files on the initialization stage. here is the most relevant code snippet: as it is const logger it somehow triggers the creation of the audit files which are some hash named files. (more about it in the other issue).

const logger = winston.createLogger({
    level: 'info', // Minimum log level to be recorded
    transports: [
        // Log errors to error.log file
        new winston.transports.DailyRotateFile({
            level: 'error',
            filename: 'logs/error-%DATE%.log',
                        lazy: true, //this had no effect.
            datePattern: 'YYYY-MM-DD',
            maxSize: '10m', // Max size of log file
            maxFiles: '14d', // Keep logs for 14 days
            format: myFilter(),
        }),
        // Log info messages to info.log file
        new winston.transports.DailyRotateFile({
            level: 'info',
            filename: 'logs/info-%DATE%.log',
                        lazy: true, //this had no effect.
            datePattern: 'YYYY-MM-DD',
            maxSize: '10m',
            maxFiles: '14d',
            format: myFilter(infoFilter)
        }),
    ],
});

Feel free to move the discussion to the other issue #378 as this might be related to personal code.