[CI] MixedClusterClientYamlTestSuiteIT test {p0=indices.split/40_routing_partition_size/more than 1} failing

elasticsearchmachine commented 2 hours ago

Build Scans:

Reproduction Line:

./gradlew ":qa:mixed-cluster:v8.16.0#mixedClusterTest" -Dtests.class="org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT" -Dtests.method="test {p0=indices.split/40_routing_partition_size/more than 1}" -Dtests.seed=B135AEB47F705106 -Dtests.bwc=true -Dtests.locale=he-Hebr-IL -Dtests.timezone=Africa/Conakry -Druntime.java=22

Applicable branches: main

Reproduces locally?: N/A

Failure History: See dashboard&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('test%20%7Bp0%3Dindices.split%2F40_routing_partition_size%2Fmore%20than%201%7D'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium)))))

Failure Message:

java.lang.AssertionError: Failure at [indices.split/40_routing_partition_size:2]: expected [2xx] status code but api [indices.create] returned [400 Bad Request] [{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"routing partition size [2] should be a positive number less than the number of shards [2] for [source]","stack_trace":"org.elasticsearch.ElasticsearchException$1: routing partition size [2] should be a positive number less than the number of shards [2] for [source]\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:706)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.ElasticsearchException.generateFailureXContent(ElasticsearchException.java:634)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.rest.RestResponse.build(RestResponse.java:187)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.rest.RestResponse.<init>(RestResponse.java:146)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.rest.RestResponse.<init>(RestResponse.java:117)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.rest.action.RestActionListener.onFailure(RestActionListener.java:54)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListener$3.onFailure(ActionListener.java:410)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.tasks.TaskManager$1.onFailure(TaskManager.java:215)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.DelegatingActionListener.onFailure(DelegatingActionListener.java:32)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$RunBeforeActionListener.onFailure(ActionListenerImplementations.java:346)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListener$3.onFailure(ActionListener.java:410)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.DelegatingActionListener.onFailure(DelegatingActionListener.java:32)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$RunBeforeActionListener.onFailure(ActionListenerImplementations.java:346)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.ActionListener$3.onFailure(ActionListener.java:410)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.action.support.master.TransportMasterNodeAction$AsyncSingleAction$1.handleException(TransportMasterNodeAction.java:278)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1509)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundHandler.doHandleException(InboundHandler.java:475)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundHandler.handleException(InboundHandler.java:462)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundHandler.handlerResponseError(InboundHandler.java:453)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundHandler.executeResponseHandler(InboundHandler.java:147)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundHandler.messageReceived(InboundHandler.java:124)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundHandler.inboundMessage(InboundHandler.java:98)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.TcpTransport.inboundMessage(TcpTransport.java:822)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundPipeline.forwardFragments(InboundPipeline.java:125)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:97)\n\tat org.elasticsearch.server@8.16.0-SNAPSHOT/org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:62)\n\tat org.elasticsearch.transport.netty4@8.16.0-SNAPSHOT/org.elasticsearch.transport.netty4.Netty4MessageInboundHandler.channelRead(Netty4MessageInboundHandler.java:58)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.codec@4.1.109.Final/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652)\n\tat io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)\n\tat io.netty.common@4.1.109.Final/io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\tat io.netty.common@4.1.109.Final/io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\tat java.base/java.lang.Thread.run(Thread.java:1570)\nCaused by: java.lang.IllegalArgumentException: routing partition size [2] should be a positive number less than the number of shards [2] for [source]\n\tat org.elasticsearch.cluster.metadata.IndexMetadata$Builder.build(IndexMetadata.java:2272)\n\tat org.elasticsearch.cluster.metadata.IndexMetadata$Builder.build(IndexMetadata.java:2246)\n\tat org.elasticsearch.xpack.logsdb.SyntheticSourceIndexSettingsProvider.buildIndexMetadataForMapperService(SyntheticSourceIndexSettingsProvider.java:127)\n\tat org.elasticsearch.xpack.logsdb.SyntheticSourceIndexSettingsProvider.newIndexHasSyntheticSourceUsage(SyntheticSourceIndexSettingsProvider.java:82)\n\tat org.elasticsearch.xpack.logsdb.SyntheticSourceIndexSettingsProvider.getAdditionalIndexSettings(SyntheticSourceIndexSettingsProvider.java:62)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService.aggregateIndexSettings(MetadataCreateIndexService.java:1004)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:602)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:441)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:320)\n\tat org.elasticsearch.cluster.service.MasterService$UnbatchedExecutor.execute(MasterService.java:563)\n\tat org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1064)\n\tat org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1027)\n\tat org.elasticsearch.cluster.service.MasterService.executeAndPublishBatch(MasterService.java:234)\n\tat org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.lambda$run$2(MasterService.java:1680)\n\tat org.elasticsearch.action.ActionListener.run(ActionListener.java:452)\n\tat org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.run(MasterService.java:1677)\n\tat org.elasticsearch.cluster.service.MasterService$5.lambda$doRun$0(MasterService.java:1272)\n\tat org.elasticsearch.action.ActionListener.run(ActionListener.java:452)\n\tat org.elasticsearch.cluster.service.MasterService$5.doRun(MasterService.java:1251)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:992)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.lang.Thread.run(Thread.java:1570)\n"}],"type":"illegal_argument_exception","reason":"routing partition size [2] should be a positive number less than the number of shards [2] for [source]","stack_trace":"java.lang.IllegalArgumentException: routing partition size [2] should be a positive number less than the number of shards [2] for [source]\n\tat org.elasticsearch.cluster.metadata.IndexMetadata$Builder.build(IndexMetadata.java:2272)\n\tat org.elasticsearch.cluster.metadata.IndexMetadata$Builder.build(IndexMetadata.java:2246)\n\tat org.elasticsearch.xpack.logsdb.SyntheticSourceIndexSettingsProvider.buildIndexMetadataForMapperService(SyntheticSourceIndexSettingsProvider.java:127)\n\tat org.elasticsearch.xpack.logsdb.SyntheticSourceIndexSettingsProvider.newIndexHasSyntheticSourceUsage(SyntheticSourceIndexSettingsProvider.java:82)\n\tat org.elasticsearch.xpack.logsdb.SyntheticSourceIndexSettingsProvider.getAdditionalIndexSettings(SyntheticSourceIndexSettingsProvider.java:62)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService.aggregateIndexSettings(MetadataCreateIndexService.java:1004)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequestWithV1Templates(MetadataCreateIndexService.java:602)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService.applyCreateIndexRequest(MetadataCreateIndexService.java:441)\n\tat org.elasticsearch.cluster.metadata.MetadataCreateIndexService$1.execute(MetadataCreateIndexService.java:320)\n\tat org.elasticsearch.cluster.service.MasterService$UnbatchedExecutor.execute(MasterService.java:563)\n\tat org.elasticsearch.cluster.service.MasterService.innerExecuteTasks(MasterService.java:1064)\n\tat org.elasticsearch.cluster.service.MasterService.executeTasks(MasterService.java:1027)\n\tat org.elasticsearch.cluster.service.MasterService.executeAndPublishBatch(MasterService.java:234)\n\tat org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.lambda$run$2(MasterService.java:1680)\n\tat org.elasticsearch.action.ActionListener.run(ActionListener.java:452)\n\tat org.elasticsearch.cluster.service.MasterService$BatchingTaskQueue$Processor.run(MasterService.java:1677)\n\tat org.elasticsearch.cluster.service.MasterService$5.lambda$doRun$0(MasterService.java:1272)\n\tat org.elasticsearch.action.ActionListener.run(ActionListener.java:452)\n\tat org.elasticsearch.cluster.service.MasterService$5.doRun(MasterService.java:1251)\n\tat org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:992)\n\tat org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)\n\tat java.lang.Thread.run(Thread.java:1570)\n"},"status":400}]

Issue Reasons:

[main] 5 consecutive failures in test test {p0=indices.split/40_routing_partition_size/more than 1}
[main] 4 consecutive failures in step 8.16.0_bwc-snapshots
[main] 2 consecutive failures in pipeline elasticsearch-intake
[main] 4 consecutive failures in pipeline elasticsearch-pull-request
[main] 18 failures in test test {p0=indices.split/40_routing_partition_size/more than 1} (3.0% fail rate in 604 executions)
[main] 17 failures in step 8.16.0_bwc-snapshots (3.2% fail rate in 533 executions)
[main] 3 failures in pipeline elasticsearch-intake (1.8% fail rate in 165 executions)
[main] 13 failures in pipeline elasticsearch-pull-request (3.4% fail rate in 383 executions)

Note: This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

elasticsearchmachine commented 2 hours ago

This has been muted on branch main

Mute Reasons:

[main] 14 failures in test test {p0=indices.split/40_routing_partition_size/more than 1} (2.3% fail rate in 600 executions)
[main] 14 failures in step 8.16.0_bwc-snapshots (2.6% fail rate in 530 executions)
[main] 2 failures in pipeline elasticsearch-intake (1.2% fail rate in 164 executions)
[main] 10 failures in pipeline elasticsearch-pull-request (2.6% fail rate in 380 executions)

Build Scans:

elasticsearchmachine commented 2 hours ago

Pinging @elastic/es-delivery (Team:Delivery)

elastic / elasticsearch

[CI] MixedClusterClientYamlTestSuiteIT test {p0=indices.split/40_routing_partition_size/more than 1} failing #113841