search

elastic / logstash-filter-elastic_integration

The Elastic Integrations filter for Logstash, which enables running Elastic Integrations inside of Logstash pipelines
Other
5 stars 8 forks source link

TODO: respond to 8.16 breaking changes in upstream ingest-geoip interfaces #167

Open yaauie opened 1 week ago

yaauie commented 1 week ago

Elasticsearch 8.16 brings breaking changes to interfaces that the ingest-geoip needs for lookups elastic/elasticsearch#114250, which is a good thing because the new boundaries allow this plugin to stop caring about the specific database models that are supported by the ingest-geoip pipelines.

We need to respond to to those changes.

I have a spike diff doing so that I believe has only one gap; it needs to ref-count the instances of IpDatabase that it hands out and ensure that only the last actually closes the underlying reader.

yaauie commented 19 hours ago

The previously-attached diff builds, but our IpDatabase#close is sent every time that the processor is done with it, so we need to add in some form of ref-counting to only close it when it's actually done. My diff just comments out the bit in IpDatabaseAdapter#close that closes the reader, which isn't an actual solution even if it makes everything work because as databases are updated or pipelines are reloaded their readers will remain. Without ref-counting, an easier path would be to add a IpDatabaseAdapter#closeReader() method that we call from the IpDatabaseHolder#close method, keeping the IpDatabaseAdapter#close method a no-op.