fail_on_missing_logfile for missing directories

[jchia]

I'm using wildcards for input paths. Even with fail_on_missing_logfile: false, if an input path's directory is missing, grok_exporter fails during startup with an error message "error reading log lines: "/some/dir": no such directory", where "/some/dir" is the directory portion of the input path.

In some use cases, not only does grok_exporter start before the file is created, it starts before the containing directory is created.

Can grok_exporter be changed to allow missing directories when fail_on_missing_logfile is false? (But if the directories start to exist, they start to be monitored for relevant input files.)

Alternatively, can there be a fail_on_missing_logdir to control allowing missing log directories?

[OperationZen]

Plus 1 on this feature request. We have a use case in which a process dynamically creates a directory in which our logfiles get generated that we want grok exporter to read. Right now I'm trying to do something with a sym_link that I would have to change very regularly.

*edit: I see that this has already been requested a couple of times, ticket #81 and #89

[gdsotirov]

@jchia Was this completed actually? I'm using latest version:

# grok_exporter --version
grok_exporter version: 1.0.0.RC5 (build date: 2020-09-21, branch: master, revision: c2f8f34, go version: go1.15.2, platform: linux-amd64)

and I'm having exactly the same error as reported in this issue:

2022-06-10 06:12:31.106624 Starting server on http://localhost:9144/metrics
2022-06-10 06:12:31.106910 error reading log lines: "/var/log/some_dir/*.jog": no such directory

I find it really annoying, because a missing directory or a missing log file should not fail the whole configuration (presumably with many other logs).

[jchia]

@gdsotirov I haven't tried newer versions since opening this issue.