search

elastic / elasticsearch

Free and Open Source, Distributed, RESTful Search Engine
https://www.elastic.co/products/elasticsearch
Other
69.62k stars 24.64k forks source link

[CI] co.elastic.elasticsearch.qa.rest.ServerlessXpackRestIT failing on remote node dying #107052

Closed bpintea closed 5 months ago

bpintea commented 5 months ago

CI Link

https://gradle-enterprise.elastic.co/s/6m6hhyu2otlxg

Repro line

./gradlew ':qa:x-pack-rest-test:yamlRestTest' --tests "co.elastic.elasticsearch.qa.rest.ServerlessXpackRestIT.test {yaml=ml/frequent_item_sets_agg/Test frequent item sets array fields profile}" -Dtests.seed=68E7B7F263D73E2 -Dtests.locale=pt-BR -Dtests.timezone=Asia/Seoul -Druntime.java=22

Does it reproduce?

Didn't try

Applicable branches

main

Failure history

No response

Failure excerpt

The local trace is provided below, but the reason for the failure is a search node dying on the following assertion (from 4068/qa/x-pack-rest-test/build/testrun/yamlRestTest/temp/yamlRestTest2243386628354282089/search-lzsLDvj/logs/yamlRestTest.log in bundle):

[2024-04-03T21:18:32,508][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [search-lzsLDvj] fatal error in thread [elasticsearch[search-lzsLDvj][search][T#15]], exiting
java.lang.AssertionError: item id exceed number of given items, did you configure eclat correctly?
    at org.elasticsearch.xpack.ml.aggs.frequentitemsets.FrequentItemSetCollector$FrequentItemSetCandidate.toFrequentItemSet(FrequentItemSetCollector.java:180) ~[?:?]
    at org.elasticsearch.xpack.ml.aggs.frequentitemsets.FrequentItemSetCollector.finalizeAndGetResults(FrequentItemSetCollector.java:262) ~[?:?]
    at org.elasticsearch.xpack.ml.aggs.frequentitemsets.EclatMapReducer.eclat(EclatMapReducer.java:572) ~[?:?]
    at org.elasticsearch.xpack.ml.aggs.frequentitemsets.EclatMapReducer.reduceFinalize(EclatMapReducer.java:389) ~[?:?]
    at org.elasticsearch.xpack.ml.aggs.frequentitemsets.EclatMapReducer.reduceFinalize(EclatMapReducer.java:69) ~[?:?]
    at org.elasticsearch.xpack.ml.aggs.frequentitemsets.mr.InternalItemSetMapReduceAggregation$1.get(InternalItemSetMapReduceAggregation.java:134) ~[?:?]
    at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212) ~[?:?]
    at java.util.HashMap$ValueSpliterator.forEachRemaining(HashMap.java:1787) ~[?:?]
    at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556) ~[?:?]
    at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546) ~[?:?]
    at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:622) ~[?:?]
    at java.util.stream.AbstractPipeline.evaluateToArrayNode(AbstractPipeline.java:291) ~[?:?]
    at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:631) ~[?:?]
    at java.util.stream.ReferencePipeline.toArray(ReferencePipeline.java:637) ~[?:?]
    at java.util.stream.ReferencePipeline.toList(ReferencePipeline.java:642) ~[?:?]
    at org.elasticsearch.search.aggregations.AggregatorsReducer.get(AggregatorsReducer.java:48) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.search.aggregations.InternalAggregations.reduce(InternalAggregations.java:285) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.search.aggregations.InternalAggregations.topLevelReduce(InternalAggregations.java:234) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.search.aggregations.InternalAggregations.topLevelReduceDelayable(InternalAggregations.java:222) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.action.search.SearchPhaseController.reduceAggs(SearchPhaseController.java:669) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.action.search.SearchPhaseController.reducedQueryPhase(SearchPhaseController.java:632) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.action.search.QueryPhaseResultConsumer.reduce(QueryPhaseResultConsumer.java:139) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.action.search.FetchSearchPhase.innerRun(FetchSearchPhase.java:99) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.action.search.FetchSearchPhase$1.doRun(FetchSearchPhase.java:87) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:984) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) ~[elasticsearch-8.14.0-SNAPSHOT.jar:?]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
    at java.lang.Thread.run(Thread.java:1570) ~[?:?]

Local failure:

REPRODUCE WITH: ./gradlew ':qa:x-pack-rest-test:yamlRestTest' --tests "co.elastic.elasticsearch.qa.rest.ServerlessXpackRestIT.test {yaml=ml/frequent_item_sets_agg/Test frequent item sets array fields profile}" -Dtests.seed=68E7B7F263D73E2 -Dtests.locale=pt-BR -Dtests.timezone=Asia/Seoul -Druntime.java=22

co.elastic.elasticsearch.qa.rest.ServerlessXpackRestIT > test {yaml=ml/frequent_item_sets_agg/Test frequent item sets array fields profile} FAILED
    java.lang.AssertionError: Failure at [ml/frequent_item_sets_agg:200]: expected [2xx] status code but api [search] returned [500 Internal Server Error] [---
    error:
      root_cause:
      - type: "node_not_connected_exception"
        reason: "[search-lzsLDvj][127.0.0.1:45633] Node not connected"
        stack_trace: "org.elasticsearch.transport.NodeNotConnectedException: [search-lzsLDvj][127.0.0.1:45633]\
          \ Node not connected\n\tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.transport.ClusterConnectionManager.getConnection(ClusterConnectionManager.java:283)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.transport.TransportService.getConnection(TransportService.java:876)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.SearchTransportService.getConnection(SearchTransportService.java:606)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.TransportSearchAction.lambda$buildConnectionLookup$17(TransportSearchAction.java:1211)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.AbstractSearchAsyncAction.getConnection(AbstractSearchAsyncAction.java:749)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.SearchQueryThenFetchAsyncAction.executePhaseOnShard(SearchQueryThenFetchAsyncAction.java:92)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.AbstractSearchAsyncAction.lambda$performPhaseOnShard$5(AbstractSearchAsyncAction.java:320)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.AbstractSearchAsyncAction.performPhaseOnShard(AbstractSearchAsyncAction.java:356)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.AbstractSearchAsyncAction.run(AbstractSearchAsyncAction.java:248)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.AbstractSearchAsyncAction.executePhase(AbstractSearchAsyncAction.java:454)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.AbstractSearchAsyncAction.start(AbstractSearchAsyncAction.java:223)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.TransportSearchAction.executeSearch(TransportSearchAction.java:1175)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.TransportSearchAction.executeLocalSearch(TransportSearchAction.java:923)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.TransportSearchAction.lambda$executeRequest$6(TransportSearchAction.java:343)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:109)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.index.query.Rewriteable.rewriteAndFetch(Rewriteable.java:77)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.TransportSearchAction.executeRequest(TransportSearchAction.java:456)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:310)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.search.TransportSearchAction.doExecute(TransportSearchAction.java:114)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:96)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:93)\n\
          \tat org.elasticsearch.serverless.apifiltering@8.14.0-SNAPSHOT/co.elastic.elasticsearch.api.validation.PublicSettingsValidationActionFilter.apply(PublicSettingsValidationActionFilter.java:110)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:93)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.ActionFilter$Simple.apply(ActionFilter.java:53)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:93)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.MappedActionFilters$MappedFilterChain.proceed(MappedActionFilters.java:70)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.MappedActionFilters.apply(MappedActionFilters.java:48)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:93)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$3(SecurityActionFilter.java:165)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$DelegatingFailureActionListener.onResponse(ActionListenerImplementations.java:217)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:637)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.SearchRequestInterceptor.intercept(SearchRequestInterceptor.java:21)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.DlsFlsLicenseRequestInterceptor.intercept(DlsFlsLicenseRequestInterceptor.java:106)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.BulkShardRequestInterceptor.intercept(BulkShardRequestInterceptor.java:85)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.UpdateRequestInterceptor.intercept(UpdateRequestInterceptor.java:27)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.SearchRequestCacheDisablingInterceptor.intercept(SearchRequestCacheDisablingInterceptor.java:53)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.IndicesAliasesRequestInterceptor.intercept(IndicesAliasesRequestInterceptor.java:124)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.ResizeRequestInterceptor.intercept(ResizeRequestInterceptor.java:98)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.ValidateRequestInterceptor.intercept(ValidateRequestInterceptor.java:20)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:635)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$1.onResponse(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.FieldAndDocumentLevelSecurityRequestInterceptor.intercept(FieldAndDocumentLevelSecurityRequestInterceptor.java:79)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.interceptor.ShardSearchRequestInterceptor.intercept(ShardSearchRequestInterceptor.java:24)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService.runRequestInterceptors(AuthorizationService.java:631)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService.handleIndexActionAuthorizationResult(AuthorizationService.java:616)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorizeAction$13(AuthorizationService.java:514)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:1030)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$AuthorizationResultListener.onResponse(AuthorizationService.java:996)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:32)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$authorizeIndexAction$3(RBACEngine.java:420)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener$SuccessResult.complete(SubscribableListener.java:378)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener.tryComplete(SubscribableListener.java:298)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener.addListener(SubscribableListener.java:201)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.SubscribableListener.addListener(SubscribableListener.java:170)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService$CachingAsyncSupplier.getAsync(AuthorizationService.java:1076)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.RBACEngine.authorizeIndexAction(RBACEngine.java:388)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService.authorizeAction(AuthorizationService.java:507)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService.maybeAuthorizeRunAs(AuthorizationService.java:439)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService.lambda$authorize$3(AuthorizationService.java:326)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:171)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.ContextPreservingActionListener.onResponse(ContextPreservingActionListener.java:32)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.RBACEngine.lambda$resolveAuthorizationInfo$0(RBACEngine.java:154)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRoles$4(CompositeRolesStore.java:193)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.lambda$getRole$5(CompositeRolesStore.java:211)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.xcore@8.14.0-SNAPSHOT/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$0(RoleReferenceIntersection.java:49)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.GroupedActionListener.onResponse(GroupedActionListener.java:56)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.buildRoleFromRoleReference(CompositeRolesStore.java:291)\n\
          \tat org.elasticsearch.xcore@8.14.0-SNAPSHOT/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.lambda$buildRole$1(RoleReferenceIntersection.java:53)\n\
          \tat java.base/java.lang.Iterable.forEach(Iterable.java:75)\n\tat org.elasticsearch.xcore@8.14.0-SNAPSHOT/org.elasticsearch.xpack.core.security.authz.store.RoleReferenceIntersection.buildRole(RoleReferenceIntersection.java:53)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRole(CompositeRolesStore.java:209)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.store.CompositeRolesStore.getRoles(CompositeRolesStore.java:186)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.RBACEngine.resolveAuthorizationInfo(RBACEngine.java:150)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authz.AuthorizationService.authorize(AuthorizationService.java:342)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.lambda$applyInternal$4(SecurityActionFilter.java:161)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$MappedActionListener.onResponse(ActionListenerImplementations.java:95)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authc.AuthenticatorChain.authenticate(AuthenticatorChain.java:93)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:264)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:173)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.applyInternal(SecurityActionFilter.java:157)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.action.filter.SecurityActionFilter.apply(SecurityActionFilter.java:114)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:93)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:68)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.tasks.TaskManager.registerAndExecute(TaskManager.java:196)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.client.internal.node.NodeClient.executeLocally(NodeClient.java:105)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.action.RestCancellableNodeClient.doExecute(RestCancellableNodeClient.java:81)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.client.internal.support.AbstractClient.execute(AbstractClient.java:356)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.action.search.RestSearchAction.lambda$prepareRequest$2(RestSearchAction.java:140)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:106)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:452)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.RestController$1.onResponse(RestController.java:446)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.rest.SecurityRestFilter.doHandleRequest(SecurityRestFilter.java:89)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.rest.SecurityRestFilter.lambda$intercept$0(SecurityRestFilter.java:81)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:171)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.lambda$authenticateAndAttachToContext$3(SecondaryAuthenticator.java:99)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.action.ActionListenerImplementations$ResponseWrappingActionListener.onResponse(ActionListenerImplementations.java:245)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticate(SecondaryAuthenticator.java:109)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.authc.support.SecondaryAuthenticator.authenticateAndAttachToContext(SecondaryAuthenticator.java:90)\n\
          \tat org.elasticsearch.security@8.14.0-SNAPSHOT/org.elasticsearch.xpack.security.rest.SecurityRestFilter.intercept(SecurityRestFilter.java:75)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:446)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:606)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:329)\n\
          \tat org.elasticsearch.serverless.rest@8.14.0-SNAPSHOT/co.elastic.elasticsearch.serverless.rest.ServerlessRestController.dispatchRequest(ServerlessRestController.java:131)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.http.AbstractHttpServerTransport.dispatchRequest(AbstractHttpServerTransport.java:487)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.http.AbstractHttpServerTransport.handleIncomingRequest(AbstractHttpServerTransport.java:583)\n\
          \tat org.elasticsearch.server@8.14.0-SNAPSHOT/org.elasticsearch.http.AbstractHttpServerTransport.incomingRequest(AbstractHttpServerTransport.java:460)\n\
          \tat org.elasticsearch.transport.netty4@8.14.0-SNAPSHOT/org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.handlePipelinedRequest(Netty4HttpPipeliningHandler.java:126)\n\
          \tat org.elasticsearch.transport.netty4@8.14.0-SNAPSHOT/org.elasticsearch.http.netty4.Netty4HttpPipeliningHandler.channelRead(Netty4HttpPipeliningHandler.java:116)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\
          \tat io.netty.codec@4.1.107.Final/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\n\
          \tat io.netty.codec@4.1.107.Final/io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\
          \tat io.netty.codec@4.1.107.Final/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\
          \tat io.netty.codec@4.1.107.Final/io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)\n\
          \tat org.elasticsearch.transport.netty4@8.14.0-SNAPSHOT/org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.forwardData(Netty4HttpHeaderValidator.java:194)\n\
          \tat org.elasticsearch.transport.netty4@8.14.0-SNAPSHOT/org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.forwardFullRequest(Netty4HttpHeaderValidator.java:137)\n\
          \tat org.elasticsearch.transport.netty4@8.14.0-SNAPSHOT/org.elasticsearch.http.netty4.Netty4HttpHeaderValidator.lambda$requestStart$1(Netty4HttpHeaderValidator.java:120)\n\
          \tat io.netty.common@4.1.107.Final/io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98)\n\
          \tat io.netty.common@4.1.107.Final/io.netty.util.concurrent.PromiseTask.run(PromiseTask.java:106)\n\
          \tat io.netty.common@4.1.107.Final/io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:173)\n\
          \tat io.netty.common@4.1.107.Final/io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:166)\n\
          \tat io.netty.common@4.1.107.Final/io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)\n\
          \tat io.netty.transport@4.1.107.Final/io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:566)\n\
          \tat io.netty.common@4.1.107.Final/io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)\n\
          \tat io.netty.common@4.1.107.Final/io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)\n\
          \tat java.base/java.lang.Thread.run(Thread.java:1570)\n"
      type: "search_phase_execution_exception"
      reason: "all shards failed"
      phase: "query"
      grouped: true
elasticsearchmachine commented 5 months ago

Pinging @elastic/ml-core (Team:ML)

davidkyle commented 5 months ago

Closing this issue and migrating to the correct repo