Windows file reader file locking/rotation issue

[ewileycoy]

Hello, I'm using Collector 0.4.0 on Windows 2008 R2 SP1 x64, Oracle jre 1.8.0_60 (win,x64). I was able to setup file-based logging for the DHCP service on windows, and it correctly tailed my logs for the first day and I got events in graylog. However when the DHCP service rotated its logs collector stopped reading, and DHCP service was no longer able to write to the next day's logs (they remained timestamped at the previous rotation date and were locked, unable to open with Notepad).

I do a search in process explorer and see the graylog collector with an open file handle on the log files (as expected), so I'm wondering if there's some best practice or other config that can be done with Windows file readers to prevent the lock from interfering with Windows rotating the logs?

I've replicated this on 3 different servers, all with the same configuration. I'm not sure this is an issue with Collector more than an issue with the way Windows DHCP service is handling log file locking, but any help would be appreciated.

This is my configuration below:

inputs {
DHCP-logs {
type = "file"
path-glob-root = "c:\\windows\\system32\\dhcp"
path-glob-pattern = "DhcpSrvLog-*.log"
content-splitter = "NEWLINE"
poll-interval = "1s"
outputs = "gelf-file"
}}
outputs {
gelf-file{
type="gelf"
host = "<my graylog server ip>"
port = 12202
}}

[bernd]

Thank you for the report. We will try to reproduce and fix it.

[MadPB]

We may have a similar issue. We have a service that logs to a file that is rotated at midnight. The Windows 0.4.1 collector maintains an open handle on the file, and at midnight when the old log is renamed with that day's date, the handle stays open to the now old file, and doesn't re-open a new handle to the new log file.

The only "fix" is to restart the graylog collector service so it picks up the new files/opens a new handle on those. I thought I saw something in a release note about the possibility of missing a file rename operation like that, but we're definitely seeing this issue.

Java 1.8.0_60, Graylog Collector for Windows 0.4.1, Windows Server 2012 R2.

Relevant config entry: offline-task-trace { type = "file" path-glob-root = "C:\OfflineTools" path-glob-pattern = "**\trace.log" content-splitter = "PATTERN" content-splitter-pattern = "(ERROR|WARN|INFO)\s\d{1,4} \d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2},\d{3}\s" }

The logging is from Log4Net tied into the app with this rotation method:

We're new to Graylog and it's going well in our testing so far except for this pretty big sticking point.

[tehpanta]

Same is happening in my scenario. RedHat/AIX mostly.

Application is unable to correctly rotate logfiles and keeps logging to rotated logfile (quite a trouble honestly, because we archive rotated logs away and delete them periodically).

[GusPS]

Same problem here with W2003, Collector 0.4.1, JRE 1.7.0. Agent and Edge Exchange Log which seems to rotate at midnight.

[Arlineas]

Anyone find a solution ?

I have the same problem when logging dhcp log on my two DHCP server

[eduardohki]

Same Problem here with graylog-collector 0.4.1 and Windows Server 2012 R2 with Exchange Server 2013.

[o1e9]

The problem remains in 0.4.2, the issue makes it difficult to collect log files from multiple applications if log file needs to be moved or re-created. I am running multiple FLEXlm daemons on Windows Server 2008R2 so have to stop graylog prior FLEXlm daemon/service restart.

[jt55401]

Same issue here on Windows 2008 R2. (Collector 0.4.2)

[jt55401]

Looks like this may not be fixed?

See the latest commit... graylog-collector is being deprecated in favor of: https://github.com/Graylog2/collector-sidecar

[joschi]

@jt55401 Yes, we'll phase out the Graylog Collector. I'd recommend using nxlog or winlogbeat to collect Windows Eventlogs and nxlog or filebeat to collect files.