yaauie
commented
4 years ago While chasing down the incorrect chaining of exceptions should definitely happen, I believe we have lower hanging fruit for this plugin.
If we change this filter to not use exceptions for lookup misses (as opposed to lookup failures like timeouts or nameserver connection errors), which will drastically reduce the number of exceptions handled and at least slow the rate of the memory leak. I have filed a ticket to track that work: https://github.com/logstash-plugins/logstash-filter-dns/issues/60
We have seen multiple reports of a memory leak, and have so far traced it down to the exceptions being raised by the underlying
Resolvlibrary when it cannot perform a lookup or a reverse-lookup.An analysis of the memory dump finds that the exceptions raised by the resolv library and immediately caught by this plugin are not being dereferenced as we would expect them to, and end up flagged as the cause of a subsequent exception, producing an endless chain from the most recent exception to the first.
I believe the root issue to be beyond the scope of this plugin, as the exceptions are
rescue'd and immediately allowed to fall out of scope, but that the volume of exceptions raised by this implementation when it is failing many lookups may exacerbate the issue.Reports: