search

mschout / perl-log-dispatch-filerotate

Log to files that archive/rotate themselves
2 stars 8 forks source link

fails to write completely if target file on NFS drive, due to locking mechanism [rt.cpan.org #41065] #16

Open mschout opened 6 years ago

mschout commented 6 years ago

https://rt.cpan.org/Ticket/Display.html?id=41065

(I'm reporting something that I noticed and looked into several months
ago, but didn't have time then to report. As such, I now don't have
fresh memory nor exhaustive description, for which I apologize.)

In my experience, FileRotate does not allow to write a log file over an
NFS mount. The reason is that the lock mechanism used by FileRotate,
fails over NFS.

If I remember well, the file is created, but remains empty (size 0)

I was using FileRotate v1.13 at the time, on a Sun Solaris 9 system, and
Perl 5.8.2. I haven't seen any mention in the changelog, so I suppose
it's still the same in the current v1.19.

When I read up on this issue several months back, I came to understand
that file locking over NFS is a very difficult issue, so this may not be
(easily) solvable. But documenting it may help other users.
mschout commented 6 years ago

just to emphasize: I'm talking about a mounted NFS drive. (so not local disk)

davehodg commented 1 year ago

Five years later, this smells like a problem we're having.

Edit: apparently it only happens on the live box, and the file system being written is "local" whatever that means in the cloud.

mschout commented 1 year ago

@davehodg sadly, the way Log::Dispatch::FileRotate works requires a lot of locking. If it is an option, maybe consider an alternative like Log::Dispatch::File::Stamped which does not require any locks whatsoever and the code is very simple.

I realize this is not a perfect answer, and it would be nice if FileRotate worked perfectly. But this particular bug is rather difficult to diagnose without a reliable way to reproduce it and possibility of broken lock semantics on differnt filesystems and environments :(

davehodg commented 1 year ago

This is academic as I no longer work there. And no, it's not reproducible, it just turns up in logs in the middle of the night. Your suspicion of NFS is probably a good one. Feel free to ignore me now :)