RotatingFileHandler formats the record twice

[serhiy-storchaka]

RotatingFileHandler calls self.format() twice for every record -- first time to determine if rollover should occur, second time to write it to the stream. It is not optimal. Even if the IO time dominates in common cases, the formatting time may be non-trivial -- this is why formatting is delayed.

It is not easy to fix without duplicating the code and introducing a coupling between several handler classes. The simplest way -- check the current file size in shouldRollover(), without adding the size of the new record. This will make the size of backups slightly larger than maxBytes, but this calculation is already inaccurate (see #64336, #69128). This will automatically solve #116263.

Linked PRs

• gh-116330
• gh-122731

[Jason-Y-Z]

Thanks for raising it, the idea makes sense to me, but I guess we might need a wider consensus on this. Nevertheless, I had a go at the implementation in https://github.com/python/cpython/pull/116330, which might help with the discussion