Rotation based on day

[fallais]

Hello,

Would you be interessed by adding the possibility to rotate log file based on day please ? If I do not any mistake, rotation is only activated by the size. The idea would be to also activate it by day : rotate everyday for example.

Thanks a lot.

[fallais]

Hello,

Little up :)

Regards.

[natefinch]

Sorry for not getting back to you sooner. I had originally thought about doing time-based rotations, but it honestly isn't that useful. What if you get 10 gigs of logs in one day and your disk is only 8 gigs? The point of rotating is to prevent you from running out of disk space. Any rotation strategy that fails that test is not one I would want to implement.

[fallais]

Well, you're right, but I really think that it is useful for investigations/forensic/SIEM. By the way, file size rotation would always be applied on top. If day rotation based is chosen and the file is exeeding the maximum size, the file could be nammed with the day and incremented (18-02-2016_......1, 18-02-2016......_2, etc..)

[natefinch]

The other problem with rotating every day is that then the amount of space your backups will take up is non-deterministic. You can have a max number of backups of 5, and a max size of 100MB... and your total size might be anywhere from 0 (if you had 5 days of rotated logs with no logging) or 500MB.

Rotating per day really just does not fit well with a sized-based model. Note that if you want to implement it yourself on top of lumberjack, there is the Rotate() method that will cause an immediate rotation... it's easy enough to write a little goroutine to rotate once per day:

log := lumberjack.Logger{ /* some config */ }
go func() {
    for {
        <-time.After(time.Hour*24)
        log.Rotate()
    }
}()

[jinleileiking]

partially agree with natefinch

[longbozhan]

a project timewriter impl daily rotate: timewriter @fallais @natefinch

[aeris]

Hello here

This issue seems block caddyserver/caddy#1096. I just escalate it as a GDPR compliance trouble and no more just as a nice to have feature.

Even outside compliance, I think this behavior should be added on this lib, given it's a decent/expectable one in all cases, as the number of related issues show, and it's at the hand of sysadmin to prefer a size or a date constraint. Yes, such feature no more ensure log space size but it's not an issue at all since ages on many other log systems.

[FelixSchwarz]

The other problem with rotating every day is that then the amount of space your backups will take up is non-deterministic. You can have a max number of backups of 5, and a max size of 100MB... and your total size might be anywhere from 0 (if you had 5 days of rotated logs with no logging) or 500MB.

I respectfully disagree that this problem should be considered a showstopper: If size differences become an issue in production, I feel that resorting to size-based rotation is just 'papering over' the underlying problem. Or to put it more diplomatically: It is a very one-sided tradeoff because it ignores other considerations.

• The ability to preserve logs for a certain amount of time. With pure size-based rotation, it is impossible to preserve at least X days of logs but no more than Y days.
• Date-based log files have helped me in the past to concentrate my focus on the relevant files with minimal tooling. In contrast, with size-based logs, I first need to check which files contain the time period I am interested in. Of course, these days most people probably use more sophisticated system-level log aggregation tools, but I like the simplicity of grep and other common Unix tools as a fallback.

That being said I completely respect your decision - thank you spending so much time in writing library in the first place.