[CI] FullClusterRestartIT testDataStreams {cluster=UPGRADED} failing

elasticsearchmachine commented 3 months ago

Build Scans:

Reproduction Line:

./gradlew ":x-pack:qa:full-cluster-restart:v8.16.0#bwcTest" -Dtests.class="org.elasticsearch.xpack.restart.FullClusterRestartIT" -Dtests.method="testDataStreams {cluster=UPGRADED}" -Dtests.seed=994157EB33BD6D28 -Dtests.bwc=true -Dtests.locale=nl-BE -Dtests.timezone=EET -Druntime.java=22

Applicable branches: main

Reproduces locally?: N/A

Failure History: See dashboard&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('testDataStreams%20%7Bcluster%3DUPGRADED%7D'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.xpack.restart.FullClusterRestartIT'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium)))))

Failure Message:

org.elasticsearch.client.ResponseException: method [GET], host [http://[::1]:41111], URI [/ds/_search], status line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [ds]","resource.type":"index_or_alias","resource.id":"ds","index_uuid":"_na_","index":"ds"}],"type":"index_not_found_exception","reason":"no such index [ds]","resource.type":"index_or_alias","resource.id":"ds","index_uuid":"_na_","index":"ds"},"status":404}

Issue Reasons:

[main] 18 consecutive failures in test testDataStreams {cluster=UPGRADED}
[main] 26 failures in test testDataStreams {cluster=UPGRADED} (4.4% fail rate in 587 executions)

Note: This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

elasticsearchmachine commented 3 months ago

This has been muted on branch main

Mute Reasons:

[main] 2 consecutive failures in test testDataStreams {cluster=UPGRADED}
[main] 10 failures in test testDataStreams {cluster=UPGRADED} (1.7% fail rate in 573 executions)

Build Scans:

elasticsearchmachine commented 3 months ago

Pinging @elastic/es-analytical-engine (Team:Analytics)

elasticsearchmachine commented 3 months ago

Pinging @elastic/es-data-management (Team:Data Management)

nik9000 commented 3 months ago

@gmarouli I think I'd like to kick this one to data management - I'm not sure what's up with the data stream not being found but it doesn't feel particularly related to the esql work I'd been looking at.

The ESQL failure had to do with a new test turning off the _field_names field in 7.xx so ESQL could verify it did the right thing when it was disabled. That test only runs in 7.x. But this seems like it's something to do with the data stream and it's everywhere. At least, at first glance it does.

elastic / elasticsearch

[CI] FullClusterRestartIT testDataStreams {cluster=UPGRADED} failing #111448