[CI] RequestBasedTaskRunnerTests testLoopOneAtATime failing

[elasticsearchmachine]

Build Scans:

• elasticsearch-intake #9326 / part2
• elasticsearch-intake #9325 / part2

Reproduction Line:

./gradlew ":x-pack:plugin:inference:test" --tests "org.elasticsearch.xpack.inference.external.http.RequestBasedTaskRunnerTests.testLoopOneAtATime" -Dtests.seed=3062999828F1030C -Dtests.locale=nb-NO -Dtests.timezone=Africa/Ndjamena -Druntime.java=22

Applicable branches: main

Reproduces locally?: N/A

Failure History: See dashboard&_a=(controlGroupInput:(chainingSystem:HIERARCHICAL,controlStyle:twoLine,ignoreParentSettings:(ignoreFilters:!f,ignoreQuery:!f,ignoreTimerange:!f,ignoreValidations:!t),panels:('0c0c9cb8-ccd2-45c6-9b13-96bac4abc542':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:task.keyword,grow:!t,id:'0c0c9cb8-ccd2-45c6-9b13-96bac4abc542',searchTechnique:wildcard,selectedOptions:!(),singleSelect:!t,title:'Gradle%20Task',width:medium),grow:!t,order:0,type:optionsListControl,width:small),'144933da-5c1b-4257-a969-7f43455a7901':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:name.keyword,grow:!t,id:'144933da-5c1b-4257-a969-7f43455a7901',searchTechnique:wildcard,selectedOptions:!('testLoopOneAtATime'),title:Test,width:medium),grow:!t,order:2,type:optionsListControl,width:medium),'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850':(explicitInput:(dataViewId:fbbdc689-be23-4b3d-8057-aa402e9ed0c5,enhancements:(),fieldName:className.keyword,grow:!t,id:'4e6ad9d6-6fdc-4fcc-bf1a-aa6ca79e0850',searchTechnique:wildcard,selectedOptions:!('org.elasticsearch.xpack.inference.external.http.RequestBasedTaskRunnerTests'),title:Suite,width:medium),grow:!t,order:1,type:optionsListControl,width:medium)))))

Failure Message:

org.mockito.exceptions.verification.TooFewActualInvocations: 
testThreadPool.executor("inference_utility");
Wanted 2 times:
-> at org.elasticsearch.xpack.inference.external.http.RequestBasedTaskRunnerTests.lambda$testLoopOneAtATime$6(RequestBasedTaskRunnerTests.java:108)
But was 1 time:
-> at org.elasticsearch.xpack.inference.external.http.RequestBasedTaskRunner.requestNextRun(RequestBasedTaskRunner.java:48)

Issue Reasons:

• [main] 2 consecutive failures in test testLoopOneAtATime
• [main] 2 consecutive failures in step part2
• [main] 2 consecutive failures in pipeline elasticsearch-intake
• [main] 2 failures in test testLoopOneAtATime (100.0% fail rate in 2 executions)
• [main] 2 failures in step part2 (100.0% fail rate in 2 executions)
• [main] 2 failures in pipeline elasticsearch-intake (100.0% fail rate in 2 executions)

Note: This issue was created using new test triage automation. Please report issues or feedback to es-delivery.

[elasticsearchmachine]

This has been muted on branch main

Mute Reasons:

• [main] 2 consecutive failures in test testLoopOneAtATime
• [main] 2 consecutive failures in step part2
• [main] 2 consecutive failures in pipeline elasticsearch-intake
• [main] 2 failures in test testLoopOneAtATime (100.0% fail rate in 2 executions)
• [main] 2 failures in step part2 (100.0% fail rate in 2 executions)
• [main] 2 failures in pipeline elasticsearch-intake (100.0% fail rate in 2 executions)

Build Scans:

• elasticsearch-intake #9326 / part2
• elasticsearch-intake #9325 / part2

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)