Sync up with ES main to fix the passing whitelists of Painless.

[mashhurs]

Description

Recent Elasticsearch source changes of Painless whitelists arrangement broke the plugin, see BK CI: https://buildkite.com/elastic/logstash-filter-elastic-integration-build/builds/138 The error we get:

48.3 > Task :compileJava
548.3 /usr/share/plugins/plugin/src/main/java/co/elastic/logstash/filters/elasticintegration/EventProcessorBuilder.java:312: error: cannot find symbol
548.3                 IngestScript.CONTEXT, PainlessPlugin.BASE_WHITELISTS,
548.3                                                     ^
548.3   symbol:   variable BASE_WHITELISTS
548.3   location: class PainlessPlugin
548.3 /usr/share/plugins/plugin/src/main/java/co/elastic/logstash/filters/elasticintegration/EventProcessorBuilder.java:313: error: cannot find symbol
548.3                 IngestConditionalScript.CONTEXT, PainlessPlugin.BASE_WHITELISTS);
548.3                                                                ^
548.3   symbol:   variable BASE_WHITELISTS
548.3   location: class PainlessPlugin
548.5 2 errors
548.6
548.6 > Task :compileJava FAILED
548.6
548.6 FAILURE: Build failed with an exception.
548.6

Now, Elasticsearch eagerly loads the base whitelists (PainlessPlugin.baseWhiteList()) and I have just tried with draft PR if we have an access for the baseWhiteList() but it seems we do not have. So it seems this requires again LogstashInternalBridge.java to access.

646.1 > Task :compileJava
646.1 /usr/share/plugins/plugin/src/main/java/co/elastic/logstash/filters/elasticintegration/EventProcessorBuilder.java:312: error: cannot find symbol
646.1                 IngestScript.CONTEXT, PainlessPlugin.baseWhiteList(),
646.1                                                     ^
646.1   symbol:   method baseWhiteList()
646.1   location: class PainlessPlugin
646.1 /usr/share/plugins/plugin/src/main/java/co/elastic/logstash/filters/elasticintegration/EventProcessorBuilder.java:313: error: cannot find symbol
646.1                 IngestConditionalScript.CONTEXT, PainlessPlugin.baseWhiteList());
646.1                                                                ^
646.1   symbol:   method baseWhiteList()
646.1   location: class PainlessPlugin
646.4 2 errors
646.5
646.5 > Task :compileJava FAILED

[yaauie]

The org.elasticsearch.ingest.LogstashInternalBridge is not dependent on org.elasticsearch.painless.spi, so it won't be as simple as adding a method as an API boundary and back-porting an implementation that calls through to PainlessPlugin::BASE_WHITELIST.

I think we have two options:

• use reflection in our code to tolerate the breaking change https://github.com/elastic/logstash-filter-elastic_integration/pull/137
• back-port the interface of PainlessPlugin.baseWhiteList() to Elasticsearch 8.13 that falls through to its own PainlessPlugin.BASE_WHITELISTS

[mashhurs]

The org.elasticsearch.ingest.LogstashInternalBridge is not dependent on org.elasticsearch.painless.spi, so it won't be as simple as adding a method as an API boundary and back-porting an implementation that calls through to PainlessPlugin::BASE_WHITELIST.

I think we have two options:

• use reflection in our code to tolerate the breaking change use reflection to get past breaking change in PainlessPlugin #137
• back-port the interface of PainlessPlugin.baseWhiteList() to Elasticsearch 8.13 that falls through to its own PainlessPlugin.BASE_WHITELISTS

Ah, thank for the investigation Ry! My opinion: from the ES issue description (... saves about 1.4M of heap, fixes concurrency issue), it sounds it is worth to backport. If backport is not possible (due to some reason), +1 for reflection approach.