search

elastic / elasticsearch-java

Official Elasticsearch Java Client
Apache License 2.0
414 stars 241 forks source link

[es/nodes.info] Failed to decode response: Missing required property 'NodeInfoXpackSecurity.http' (JSON path: nodes.<node id>.settings.xpack.security #892

Open marcreichman-pfi opened 1 week ago

marcreichman-pfi commented 1 week ago

Java API client version

8.15.2

Java version

21

Elasticsearch Version

8.15.2

Problem description

When running the following Java code, this error happens, and it seems to depend on the state of the various xpack security settings. We've seen different combinations based on if security is off, or if it's on and there are no added users, etc.. -- We have not tried every combination of server settings. There are some scenarios where it works fine as well, but where it does not, there is not any way to work around this since we cannot enforce a specific security configuration to every installation.

Java code to reproduce

Either

client.nodes().info(n -> n.nodeId("_master")).get()
client.nodes().info().get()

Example stack traces (they vary..):

co.elastic.clients.transport.TransportException: node: http://192.168.0.201:9200/, status: 200, [es/nodes.info] Failed to decode response
    at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:404) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:363) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.lambda$performRequestAsync$0(ElasticsearchTransportBase.java:211) ~[elasticsearch-java-8.15.2.jar:?]
    at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179) ~[?:?]
    at co.elastic.clients.transport.rest_client.RestClientHttpClient$1.onSuccess(RestClientHttpClient.java:115) ~[elasticsearch-java-8.15.2.jar:?]
    at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:680) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:403) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:397) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) ~[httpcore-4.4.16.jar:4.4.16]
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[httpcore-nio-4.4.13.jar:4.4.13]
    ... 1 more
Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.nodes.info.NodeInfoXpack: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'NodeInfoXpackSecurity.http' (JSON path: nodes.ESSEarAiRVKrERBXUjJPMg.settings.xpack.security) (line no=1, column no=1548, offset=-1)
    at co.elastic.clients.json.JsonpMappingException.from0(JsonpMappingException.java:134) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpMappingException.from(JsonpMappingException.java:121) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:236) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:349) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:333) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:399) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:363) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.lambda$performRequestAsync$0(ElasticsearchTransportBase.java:211) ~[elasticsearch-java-8.15.2.jar:?]
    at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179) ~[?:?]
    at co.elastic.clients.transport.rest_client.RestClientHttpClient$1.onSuccess(RestClientHttpClient.java:115) ~[elasticsearch-java-8.15.2.jar:?]
    at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:680) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:403) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:397) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) ~[httpcore-4.4.16.jar:4.4.16]
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[httpcore-nio-4.4.13.jar:4.4.13]
    ... 1 more
Caused by: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'NodeInfoXpackSecurity.http'
    at co.elastic.clients.util.ApiTypeHelper.requireNonNull(ApiTypeHelper.java:76) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.elasticsearch.nodes.info.NodeInfoXpackSecurity.<init>(NodeInfoXpackSecurity.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.elasticsearch.nodes.info.NodeInfoXpackSecurity.<init>(NodeInfoXpackSecurity.java:62) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.elasticsearch.nodes.info.NodeInfoXpackSecurity$Builder.build(NodeInfoXpackSecurity.java:242) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.elasticsearch.nodes.info.NodeInfoXpackSecurity$Builder.build(NodeInfoXpackSecurity.java:159) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:80) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:349) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:333) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:399) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:363) ~[elasticsearch-java-8.15.2.jar:?]
    at co.elastic.clients.transport.ElasticsearchTransportBase.lambda$performRequestAsync$0(ElasticsearchTransportBase.java:211) ~[elasticsearch-java-8.15.2.jar:?]
    at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179) ~[?:?]
    at co.elastic.clients.transport.rest_client.RestClientHttpClient$1.onSuccess(RestClientHttpClient.java:115) ~[elasticsearch-java-8.15.2.jar:?]
    at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:680) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:403) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:397) ~[elasticsearch-rest-client-8.15.2.jar:8.15.2]
    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122) ~[httpcore-4.4.16.jar:4.4.16]
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40) ~[httpasyncclient-4.1.5.jar:4.1.5]
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) ~[httpcore-nio-4.4.13.jar:4.4.13]
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) ~[httpcore-nio-4.4.13.jar:4.4.13]
    ... 1 more
Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.nodes.info.NodeInfoXpackSecurity: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'NodeInfoXpackSecurityAuthc.realms' (JSON path: nodes['1mF7hYmESmSU5V7yqtvy-w'].settings.xpack.security.authc) (line no=1, column no=1549, offset=-1)
Caused by: co.elastic.clients.util.MissingRequiredPropertyException: Missing required property 'NodeInfoXpackSecurityAuthc.realms'
marcreichman-pfi commented 1 week ago

I just want to follow up to indicate, that this happened with (docker run):

-e xpack.security.enabled=false

When I explicitly added this:

-e xpack.security.http.ssl.enabled=false

It worked again for the first problem.

The second problem seems to be related to having the anonymous realm created.

We also were able to get past both using the approach I found in #891:

try (ApiTypeHelper.DisabledChecksHandle h = ApiTypeHelper.DANGEROUS_disableRequiredPropertiesCheck(true)) {

This seems like not the best idea moving forward, though.

l-trotta commented 1 week ago

Hello! Thanks for reporting this. These are both errors in our API specification definition, I have already provided a fix for both of them, then we'll regenerate the client to fix the issues. The next patch has already been built, so this will probably end up in the next minor.

marcreichman-pfi commented 1 week ago

Hi @l-trotta ,

Thanks for your fix! I saw your PR for this: https://github.com/elastic/elasticsearch-specification/pull/2998

My impression is that this would fix the second of my error messages, but not the first. I do not believe it should be required to set xpack.security.http.ssl.enabled=false if xpack.security.enabled=false is already set.

I think this line should also have http?, unless I'm misunderstanding something. https://github.com/elastic/elasticsearch-specification/blob/06c57215ac9fa483f138994ccf6bf4fae78c3a49/specification/nodes/info/types.ts#L247

Thanks for your help to fix this!

l-trotta commented 1 week ago

There's another PR for that :D

karlney commented 1 week ago

Hi I have a very similar issue (on the same version of the client (8.15.2) and the same call (to es/nodes.info)

I get this

Exception in thread "Thread-4" java.util.concurrent.ExecutionException: co.elastic.clients.transport.TransportException: node: http://localhost:9201/, status: 200, [es/nodes.info] Failed to decode response
    at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:396)
    at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2073)    
...
$0(TestRunnerService.kt:80)
    at java.base/java.lang.Thread.run(Thread.java:1583)

Caused by: co.elastic.clients.transport.TransportException: node: http://localhost:9201/, status: 200, [es/nodes.info] Failed to decode response
    at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:404)
    at co.elastic.clients.transport.ElasticsearchTransportBase.getApiResponse(ElasticsearchTransportBase.java:363)
    at co.elastic.clients.transport.ElasticsearchTransportBase.lambda$performRequestAsync$0(ElasticsearchTransportBase.java:211)
    at java.base/java.util.concurrent.CompletableFuture.uniHandle(CompletableFuture.java:934)
    at java.base/java.util.concurrent.CompletableFuture$UniHandle.tryFire(CompletableFuture.java:911)
    at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
    at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2179)
    at co.elastic.clients.transport.rest_client.RestClientHttpClient$1.onSuccess(RestClientHttpClient.java:115)
    at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onSuccess(RestClient.java:680)
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:403)
    at org.elasticsearch.client.RestClient$1.completed(RestClient.java:397)
    at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:182)
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
    at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
    at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
    at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
    at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
    at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
    at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
    ... 1 more
Caused by: co.elastic.clients.json.JsonpMappingException: Error deserializing co.elastic.clients.elasticsearch.nodes.info.NodeInfoSettingsNetwork: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'START_ARRAY' instead of '[KEY_NAME, VALUE_STRING, VALUE_NUMBER, VALUE_TRUE, VALUE_FALSE]' (JSON path: nodes['6lgrxgYTTzeiKZCXyzOLYw'].settings.network.host) (line no=1, column no=1829, offset=-1)
    at co.elastic.clients.json.JsonpMappingException.from0(JsonpMappingException.java:134)
    at co.elastic.clients.json.JsonpMappingException.from(JsonpMappingException.java:121)
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:236)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157)
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157)
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157)
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
    at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:349)
    at co.elastic.clients.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:333)
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:166)
    at co.elastic.clients.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:157)
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:77)
    at co.elastic.clients.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:79)
    at co.elastic.clients.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:43)
    at co.elastic.clients.transport.ElasticsearchTransportBase.decodeTransportResponse(ElasticsearchTransportBase.java:399)
    ... 25 more
Caused by: co.elastic.clients.json.UnexpectedJsonEventException: Unexpected JSON event 'START_ARRAY' instead of '[KEY_NAME, VALUE_STRING, VALUE_NUMBER, VALUE_TRUE, VALUE_FALSE]'
    at co.elastic.clients.json.JsonpUtils.ensureAccepts(JsonpUtils.java:117)
    at co.elastic.clients.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:76)
    at co.elastic.clients.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:78)
    at co.elastic.clients.json.ObjectDeserializer.deserializeInner(ObjectDeserializer.java:214)
    ... 55 more

I can report this as a separate bug also, but they look very similar

l-trotta commented 1 week ago

hey @karlney, this seems to be another issue, looks like the host field in NodeInfoSettingsNetwork can also be an array. No need to create another issue, I'll fix this asap!

marcreichman-pfi commented 1 week ago

@l-trotta Thanks for all your prompt attention on these fixes. I am curious, is the server's configuration and API generated from these specifications as well, or does the spec just do its best to track and catch up with the actual server mechanics?

The workaround you've posted with the suppression of the field validation has been a lifesaver, but I've always been curious why this new client has been subject to so many of these types of problems.

l-trotta commented 1 week ago

@marcreichman-pfi

or does the spec just do its best to track and catch up with the actual server mechanics

this one ^^" I didn't get to work on in so I'm not sure, but I think the older client didn't try to replicate the server requests/responses as closely as we're doing now, especially not with the requires or nullable fields. So while it was harder to incur in errors like these ones, it was also much easier to craft a query that the server would not accept.

marcreichman-pfi commented 1 week ago

I wonder if the server tests are robust enough that someone could experiment with generating the server-side API or config parsing from the same specifications.

I appreciate that the HLRC and TransportClient before that brought it all kinds of heavy dependencies, but at least there wasn't a worry about being in sync with the server. Fundamentally this is a better choice, and I assume politics are involved, but the server should come from the same spec too. Some day.