Open joshuaspence opened 7 years ago
@joshuaspence I'm not against this, but I have some thoughts:
1) /etc doesn't exist on windows. 2) If this were a default, it would be a change in behavior we'd have to delay shipping with Logstash until Logstash 6.0 (but you could install it earlier) 3) If this were a default, how would a user say "Don't load the patterns from /etc" ?
Regarding Windows, maybe the Logstash core should provide a method for accessing the configuration directory (/etc
on Linux, not sure about Windows)?
Another option might be to allow plugins to be considered figured globally. As in, if I could set logstash.filters.grok.patterns_dir: '/etc/logstash/patterns'
in /etc/logstash/logstash.yml
I'm open to that. We used to have a flag (--patterns-path? I forget) but I removed it because Grok was they only plugin (out of hundreds) where there was a desire for a global setting.
Another plugin where I have wanted to set a global default is logstash-filter-geoip
, where I have wanted to set a global default for the path to the GeoIP database file.
@joshuaspence Since we have a path.settings
now in Logstash (as of 5.0), I think having a namespace available for each plugin to set defaults might be doable.
Can you open an issue about this on elastic/logstash?
Filed elastic/logstash#6609.
Couldn't we simply load an environment variable LOGSTASH_FILTER_GROK_PATTERN_DIRS, which would take a colon separated list of paths? That would require no change to logstash, is easy to inject (in case of systemd you can simply use drop-in units, or maybe you control the deployment of logstash anyway), and should require very little effort to implement.
Currently this plugin loads patterns from the
patterns/
directory withinlogstash-patterns-core
and also from$LOGSTASH_HOME/patterns
by default:On Logstash 5,
$LOGSTASH_HOME/patterns
is/usr/share/logstash/patterns
. It seems more common, however, for users to define custom grok patterns in/etc/logstash/patterns
rather than$LOGSTASH_HOME/patterns
. The officialelastic/logstash
Puppet module, for example, provides alogstash::patternfile
resource, which creates files in this directory. This works fine, but means that the user needs to addpatterns_dir => "/etc/logstash/patterns"
to everygrok
filter.It seems to me that it would be better if the
grok
filter simply loaded patterns from/etc/logstash/patterns
by default.