search

driskell / log-courier

The Log Courier Suite is a set of lightweight tools created to ship and process log files speedily and securely, with low resource usage, to Elasticsearch or Logstash instances.
Other
419 stars 107 forks source link

Windows file lock #213

Closed centran closed 8 years ago

centran commented 9 years ago

It appears your fix for the locking of files no longer works. I am having problems with logs that rotate because the file is being locked.

driskell commented 9 years ago

Can you provide some logs from Log Courier and possible from your logging application? What version of Windows are you running?

On Windows, files are opened with FILE_SHARE_READ, FILE_SHARE_WRITE and FILE_SHARE_DELETE flags. This basically means the logging application should be able to do anything to the file and not have any problems.

This area of code hasn't been touched in over a year so I would expect behaviour to be the same.

centran commented 9 years ago

It may be the logging platform we are using called nlog. I was able to rename the file perfectly fine but our application just refuses to rename and rotate the log when log-courier has the file open and reading. I have confirmed that the secondary attributes you are setting with your fix are being set by only setting read-only and then I can no longer rename the file. There is something wrong with nlog that is not rotating the file when it is being read by another program.

centran commented 9 years ago

On a side note, I notice the logstash file input plugin will look at the file modified time at a set interval and then read the file when there is a change. This is probably a better question for logstash-forwarder team but I am wondering why they choose to leave the file open and not mimic logstashs behavior.

As I am not familiar with go or how the harvester interacts with the core program I am not sure how difficult it would be to switch the behavior.

driskell commented 9 years ago

Logstash-forwarder and log courier work the same really. Just set the dead time low to a few minutes.

Closing and opening is bad behaviour as it will waste resource especially with very active log files. Keeping open is much faster. Also logstash input is a poller where forwarder and courier are event based (through the way Go works) so as soon as data is written to file we read it immediately. Keeps resource steady smooth and low.

I don't believe it to be related though as really it shouldn't affect the logging application.

As mentioned earlier do you have logs showing the error from nlog when it tries to rotate? And what version of Windows? Essentially we NEED to find out WHY nlog can't rotate - Windows will be giving it an error and that error is key to finding out what we need to do, if anything.

centran commented 9 years ago

These are Windows 2012 R2 servers. As I mentioned it is a .NET application using nlog to help write log files. I will look into if it is spitting out another log somewhere else when it fails to rotate the log and what error message it may be giving. I'll let you know when I hunt down the message.

I see the "dead time" config and that should help solve the problem I am having for the time being. This issue might also be because we are using the AsyncWriter type/module. I am just taking a wild guess that because multiple processes could be writing that nlog just sees an open file handle and doesn't rotate the log. So this may more be nlogs fault then your own as you appear to be opening the file correctly. I am not sure if there is much else you can do from your end.

driskell commented 9 years ago

Did you manage to find anything?