search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.64k stars 24.65k forks source link

[CI] MixedClusterClientYamlTestSuiteIT class failing #113466

Open elasticsearchmachine opened 2 hours ago

elasticsearchmachine commented 2 hours ago

Build Scans:

Reproduction Line:

./gradlew ":qa:mixed-cluster:v8.16.0#mixedClusterTest" -Dtests.class="org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT" -Dtests.method="test {p0=search/40_indices_boost/Indices boost with alias}" -Dtests.seed=DD0A7F60139F7BB0 -Dtests.bwc=true -Dtests.locale=sah-Cyrl-RU -Dtests.timezone=America/Santarem -Druntime.java=22

Applicable branches: main

Reproduces locally?: N/A

Failure History: See dashboard&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium)))))

Failure Message:

java.lang.AssertionError: Failure at [search/40_indices_boost:2]: expected [2xx] status code but api [indices.create] returned [503 Service Unavailable] [{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null,"stack_trace":"org.elasticsearch.discovery.MasterNotDiscoveredException\n"}],"type":"master_not_discovered_exception","reason":null,"stack_trace":"org.elasticsearch.discovery.MasterNotDiscoveredException\n"},"status":503}]

Issue Reasons:

Note: This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

elasticsearchmachine commented 2 hours ago

This has been muted on branch main

Mute Reasons:

Build Scans:

elasticsearchmachine commented 2 hours ago

Pinging @elastic/es-search-relevance (Team:Search Relevance)

benwtrent commented 2 hours ago 
[2024-09-24T13:27:32,771][ERROR][o.e.ExceptionsHelper     ] [v8.16.0-0] fatal error java.lang.AssertionError: Unknown NamedWriteable [org.elasticsearch.search.DocValueFormat][_ts_routing_hash]
    at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.ExceptionsHelper.lambda$maybeDieOnAnotherThread$4(ExceptionsHelper.java:445)
    at java.base/java.util.Optional.ifPresent(Optional.java:178)
    at org.elasticsearch.server@9.0.0-SNAPSHOT/org.elasticsearch.ExceptionsHelper.maybeDieOnAnotherThread(ExceptionsHelper.java:435)
    at org.elasticsearch.transport.netty4@9.0.0-SNAPSHOT/org.elasticsearch.transport.netty4.Netty4MessageInboundHandler.exceptionCaught(Netty4MessageInboundHandler.java:67)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:346)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:447)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.codec@4.1.109.Final/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
    at io.netty.transport@4.1.109.Final/io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
    at io.netty.transport@4.1.109.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
    at io.netty.transport@4.1.109.Final/io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
    at io.netty.transport@4.1.109.Final/io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
    at io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
    at io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:689)
    at io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:652)
    at io.netty.transport@4.1.109.Final/io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
    at io.netty.common@4.1.109.Final/io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
    at io.netty.common@4.1.109.Final/io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
    at java.base/java.lang.Thread.run(Thread.java:1570)

The mixed cluster crashed due to this. Pinging appropriate team.

elasticsearchmachine commented 2 hours ago

Pinging @elastic/es-storage-engine (Team:StorageEngine)

benwtrent commented 2 hours ago

Marking as a blocker as this muted the entire mixed cluster suite.

martijnvg commented 2 hours ago

This should be addressed by #113452 (and when it is back ported).

dnhatn commented 1 hour ago

Fixed in https://github.com/elastic/elasticsearch/pull/113452

benwtrent commented 1 hour ago

@dnhatn @martijnvg while it might be fixed, has the test suite been unmuted? I would rather not close this until we unmute the suite on the affected branches.

martijnvg commented 1 hour ago

I would rather not close this until we unmute the suite on the affected branches.

👍 the PR that unmutes the test should close this issue.