Closed mark-vieira closed 3 months ago
Pinging @elastic/es-storage-engine (Team:StorageEngine)
Looks like a test issue. During the rolling upgrade the cluster tries the allocate a replica, but fails 5 times. After which no retry is happening. Even when the upgrade is completed and the primary shard is allocated:
"my-index" : {
"shards" : {
"0" : [
{
"state" : "STARTED",
"primary" : true,
"node" : "vz9cOsqBRGSQfSLU8TOO0w",
"relocating_node" : null,
"shard" : 0,
"index" : "my-index",
"allocation_id" : {
"id" : "N-QFY6eRRQiYoLuTcdzPqg"
}
},
{
"state" : "UNASSIGNED",
"primary" : false,
"node" : null,
"relocating_node" : null,
"shard" : 0,
"index" : "my-index",
"recovery_source" : {
"type" : "PEER"
},
"unassigned_info" : {
"reason" : "ALLOCATION_FAILED",
"at" : "2024-06-06T20:29:55.535Z",
"failed_attempts" : 5,
"failed_nodes" : [
"DN1nowVMRZeeiWTNpNoxxQ",
"5SaXoi7LQv-rMGdQWoEcsg"
],
"delayed" : false,
"details" : "failed shard on node [DN1nowVMRZeeiWTNpNoxxQ]: failed to create index, failure IllegalArgumentException[Setting index.mapper.dynamic was removed after version 6.0.0]",
"allocation_status" : "no_attempt"
}
}
]
}
},
This should be fixed now.
This looks to be a new test added in https://github.com/elastic/elasticsearch/pull/109301
Build scan: https://gradle-enterprise.elastic.co/s/hoxn5f3dd54hk/tests/:qa:rolling-upgrade:v7.13.4%23oneThirdUpgradedTest/org.elasticsearch.upgrades.MappingIT/testMapperDynamicIndexSetting
Reproduction line:
Applicable branches: 7.17
Reproduces locally?: Didn't try
Failure history: Failure dashboard for
org.elasticsearch.upgrades.MappingIT#testMapperDynamicIndexSetting
&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('testMapperDynamicIndexSetting'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.upgrades.MappingIT'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium)))))Failure excerpt: