Closed autarch closed 7 years ago
From drolsky@cpan.org (@autarch) on 2015-11-13 23:16:28:
On Fri Nov 13 08:42:10 2015, SREZIC wrote:
It seems that writes done from multiple processes to a shared file are only atomic if the file was opened with O_APPEND, at least on Linux and for reasonable string lengths. The attached script seems to confirm this statement --- with '>>' I get always 1000 lines of output, with '>' it's always less than 1000.
It would be nice if this could be mentioned in the Log::Dispatch::File documentation.
There is another output, Log::Dispatch::File::Locked (https://metacpan.org/pod/Log::Dispatch::File::Locked) that is designed to share a file across processes.
I could add a pointer in the File docs to File::Locked.
From srezic@cpan.org (@eserte) on 2015-11-16 14:18:26:
On 2015-11-13 18:16:28, DROLSKY wrote:
On Fri Nov 13 08:42:10 2015, SREZIC wrote:
It seems that writes done from multiple processes to a shared file are only atomic if the file was opened with O_APPEND, at least on Linux and for reasonable string lengths. The attached script seems to confirm this statement --- with '>>' I get always 1000 lines of output, with '>' it's always less than 1000.
It would be nice if this could be mentioned in the Log::Dispatch::File documentation.
There is another output, Log::Dispatch::File::Locked (https://metacpan.org/pod/Log::Dispatch::File::Locked) that is designed to share a file across processes.
I could add a pointer in the File docs to File::Locked.
Locking is another possibility. But I think that file locks are less efficient than normal writes in O_APPEND mode (more syscalls involved, more switching between kernel and user space). Probably both approaches should be mentioned.
From drolsky@cpan.org (@autarch) on 2015-12-19 19:58:11:
On Mon Nov 16 09:18:26 2015, SREZIC wrote:
On 2015-11-13 18:16:28, DROLSKY wrote:
On Fri Nov 13 08:42:10 2015, SREZIC wrote:
It seems that writes done from multiple processes to a shared file are only atomic if the file was opened with O_APPEND, at least on Linux and for reasonable string lengths. The attached script seems to confirm this statement --- with '>>' I get always 1000 lines of output, with '>' it's always less than 1000.
It would be nice if this could be mentioned in the Log::Dispatch::File documentation.
There is another output, Log::Dispatch::File::Locked (https://metacpan.org/pod/Log::Dispatch::File::Locked) that is designed to share a file across processes.
I could add a pointer in the File docs to File::Locked.
Locking is another possibility. But I think that file locks are less efficient than normal writes in O_APPEND mode (more syscalls involved, more switching between kernel and user space). Probably both approaches should be mentioned.
Using O_APPEND and making sure that all your writes are under a certain size seems very fragile. I agree that using locks is going to be slower, but it's also not likely to just blow up entirely ;)
That said, I think any sane production app should probably be using syslog.
Migrated from rt.cpan.org #108899 (status was 'open')
Requestors:
Attachments:
From srezic@cpan.org (@eserte) on 2015-11-13 13:42:10:
It seems that writes done from multiple processes to a shared file are only atomic if the file was opened with O_APPEND, at least on Linux and for reasonable string lengths. The attached script seems to confirm this statement --- with '>>' I get always 1000 lines of output, with '>' it's always less than 1000.
It would be nice if this could be mentioned in the Log::Dispatch::File documentation.
Some pointers about atomic writes: