Incorrect mapping of files in audit.json file

[Vishal8523]

Versions: winston: ^2.4.5 winston-daily-rotate-file: ^3.10.0 Node JS: v12.22.0 OS: Debian 11 Bullseye

During the logs rotate mechanism, some of the files are not deleted from the logs directory. Upon investigation, it is found that the -audit.json file has missing files that made it to not get deleted.

Winston Settings:

const logger = new winston.Logger({
    transports: [
        new winston.transports.Console({
            level: 'info',
            colorize: true,
            prettyPrint: true,
            timestamp: true
        }),
        new winston.transports.DailyRotateFile({
            filename: Logs-%DATE%.log,
            datePattern: 'YYYY-MM-DD',
            dirname: /logs/,
            level: 'info',
            json: false,
            prettyPrint: true,
            timestamp: true,
            zippedArchive: false,
            maxSize: 100m,
            maxFiles: 7d
        })
    ]
});

The audit.json file is as on 18th Oct 2022. Here, the 17th Oct 2022 file is missing in the audit.json file and this file (Logs-2022-10-17.log) was not deleted from the logs directory.

{
    "keep": {
        "days": true,
        "amount": 7
    },
    "auditLog": "/logs/.10f0fac856acd0450d842d9db8cf031638f51968-audit.json",
    "files": [
        {
            "date": 1665546274787,
            "name": "/logs/Logs-2022-10-12.log",
            "hash": "9e4b2afc2135014d339ded5cfb4d658a"
        },
        {
            "date": 1665632751118,
            "name": "/logs/Logs-2022-10-13.log",
            "hash": "6da82625ff3ac1b2696f5cc329c6c79b"
        },
        {
            "date": 1665705699082,
            "name": "/logs/Logs-2022-10-14.log",
            "hash": "03d79ac4b065b944f801f79b9193bab9"
        },
        {
            "date": 1665792068107,
            "name": "/logs/Logs-2022-10-15.log",
            "hash": "34e6ff4052e4ca803a86b49880bf9eb0"
        },
        {
            "date": 1665878484923,
            "name": "/logs/Logs-2022-10-16.log",
            "hash": "7bb75b7fe8e6a6ab80aa092ed44a4230"
        },
        {
            "date": 1666065076652,
            "name": "/logs/Logs-2022-10-18.log",
            "hash": "88f06d511f09711d71cabb7b98e52aef"
        }
    ]
}

Requesting a resolution on the issue.

[wbt]

At first glance, it looks like the file may have been locked by some process (e.g. an automated backup) at the time the deletion was attempted.

[Vishal8523]

How to avoid this situation and have proper rotation/deletion of logs ? Considering multiple node processes are writing to the same log file

Note: Also tried removing the file manually from the audit.json but the file gets deleted.

[wbt]

Why do you have multiple Node processes writing to the same log file? You could consider having separate processes log into separate log directories and disabling automated lockers (e.g. backup processes) if you want to increase the probability of correct rotation/deletion of logs.

[Vishal8523]

It is required by the application to log only 1 file from all the Node processes. So, no much control over it. How does one disable automated lockers ?

[wbt]

The automated locking is a file-system feature. What would you expect the result to be if one process tried to write to a file while another process tried to delete it at the same time? That "requirement" as stated seems inconsistent with your use of file rotation, which is going to have multiple files. You may want to consider more deeply what the source/reasoning of that might be - maybe have different processes log to different directories and something else that can read from different directories/streams to present a merged view, or coordinate logging through a single process.