search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.94k stars 24.74k forks source link

Add mixed mode BWC testing support for test-specific plugins #115166

Open ChrisHegarty opened 3 days ago

ChrisHegarty commented 3 days ago

This issues been filed to examine how best to support the inference-service-test plugin in ES|QL mixed version testing.

The ES|QL CSV and REST tests run with a variety of modes (see x-pack/plugin/esql/qa/server): 1) single-node 2) multi-node
3) mixed-cluster 4) multi-clusters

No.1 and no.2 run the same ES version on all nodes. No.3 and no.4 run with mixed ES versions, either within the same cluster or across clusters.

The issue occurs only with no.3 and no.4 where the BWC infrastructure is not able to install the interference service test-specific plugin into the older nodes. When we try it, we get the following error:

> Task :x-pack:plugin:esql:qa:server:mixed-cluster:v8.17.0#javaRestTest FAILED

MixedClusterEsqlSpecIT > classMethod FAILED
    java.lang.RuntimeException: An error occurred orchestrating test cluster.
        at __randomizedtesting.SeedInfo.seed([27B9DA09AA60FCA0]:0)
        at org.elasticsearch.test.cluster.local.DefaultLocalClusterHandle.execute(DefaultLocalClusterHandle.java:270)
        at org.elasticsearch.test.cluster.local.DefaultLocalClusterHandle.execute(DefaultLocalClusterHandle.java:275)
        at org.elasticsearch.test.cluster.local.DefaultLocalClusterHandle.start(DefaultLocalClusterHandle.java:78)
        at org.elasticsearch.test.cluster.local.DefaultLocalElasticsearchCluster$1.evaluate(DefaultLocalElasticsearchCluster.java:46)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
        at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
        at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
        at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
        at java.base/java.lang.Thread.run(Thread.java:1570)

        Caused by:
        java.lang.IllegalArgumentException: Can't run executable: `/Users/chegar/git/elasticsearch-lucene_snapshot/x-pack/plugin/esql/qa/server/mixed-cluster/build/testrun/v8.17.0_javaRestTest/temp/test-cluster8788876539987453686/test-cluster-2/distro/bin/elasticsearch-plugin` does not exist.
            at org.elasticsearch.test.cluster.util.ProcessUtils.exec(ProcessUtils.java:54)
            at org.elasticsearch.test.cluster.local.AbstractLocalClusterFactory$Node.runToolScript(AbstractLocalClusterFactory.java:784)
            at org.elasticsearch.test.cluster.local.AbstractLocalClusterFactory$Node.installPlugins(AbstractLocalClusterFactory.java:633)
            at org.elasticsearch.test.cluster.local.AbstractLocalClusterFactory$Node.start(AbstractLocalClusterFactory.java:159)
            at org.elasticsearch.test.cluster.local.DefaultLocalClusterHandle.lambda$start$0(DefaultLocalClusterHandle.java:78)
            at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
            at java.base/java.util.AbstractList$RandomAccessSpliterator.forEachRemaining(AbstractList.java:722)
            at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
            at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:291)
            at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:759)
            at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
            at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:676)
            at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:160)
            at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:174)
            at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:264)
            at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:611)
            at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:785)
            at org.elasticsearch.test.cluster.local.DefaultLocalClusterHandle.lambda$start$1(DefaultLocalClusterHandle.java:78)
            at org.elasticsearch.test.cluster.local.DefaultLocalClusterHandle.lambda$execute$16(DefaultLocalClusterHandle.java:276)
            at java.base/java.util.concurrent.ForkJoinTask$AdaptedInterruptibleCallable.compute(ForkJoinTask.java:1689)
            at java.base/java.util.concurrent.ForkJoinTask$InterruptibleTask.exec(ForkJoinTask.java:1641)
            at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:507)
            at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1491)
            at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:2073)
            at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:2035)
            at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:187)
elasticsearchmachine commented 3 days ago

Pinging @elastic/es-search (Team:Search)

elasticsearchmachine commented 3 days ago

Pinging @elastic/es-core-infra (Team:Core/Infra)