search

logstash-plugins / logstash-filter-dns

Apache License 2.0
7 stars 28 forks source link

FR: Option to disable logging DNS lookup failures #47

Closed pwhack closed 1 year ago

pwhack commented 5 years ago

Hi,

I noticed in my Logstash (v6.4.3) logs many many instances of WARN level messages detailing all the times an IP address could not be reversed to a hostname in my pipeline that leverages logstash-filter-dns reverse lookup. Certainly this information is good to know and good default behavior but in my scenario it's fine if an IP address has no reverse name (PTR record) and I'd like the option to disable logging these reverse lookup failures.

Perhaps adding a Boolean setting called something like 'log_failures' that defaults to 'true' is the way to go?

Or if this is already controllable via Logstash's Log4J properties file, then please update logstash-filter-dns documentation (https://www.elastic.co/guide/en/logstash/current/plugins-filters-dns.html) to explain what to modify in the L4J file to accomplish this without disabling other WARN level messages.

Thanks, Patrick

pwhack commented 5 years ago

I noticed @jsvd's and @wiibaa's idea of adding a tag on lookup failures (https://github.com/logstash-plugins/logstash-filter-dns/issues/24) and I think that's better than logging to Logstash's log with these WARN messages. +1 on that issue.

SpencerLN commented 5 years ago

An option to disable the WARN messages is very necessary at high volumes, we end up filling our logs with errors for when fields do not exist.

[2019-01-24T06:23:57,645][WARN ][logstash.filters.dns ] DNS filter could not perform reverse lookup on missing field {:field=>"[destination][hostname_resolved]"}

snowline-dev commented 3 years ago

Still actual in 2020.

Without this option there are a lot of unnecessary WARN in logs and on huge amount of events it's impossible to track real, useful WARN.