search

Graylog2 / graylog2-server

Free and open log management
https://www.graylog.org
Other
7.42k stars 1.07k forks source link

Missing required property 'NodeInfoAction.destructiveRequiresName' #18619

Open patrickmann opened 8 months ago

patrickmann commented 8 months ago

When the UI initializes after login, this exception appears. It only shows up once and I don't notice any functional issues (though I can't tell if telemetry is affected).

Expected Behavior

No exception

Current Behavior

2024-03-14 08:52:24,570 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource org.graylog.shaded.opensearch2.org.opensearch.OpenSearchException: Could not read nodes list from indexer! at org.graylog.storage.opensearch2.OpenSearchClient.exceptionFrom(OpenSearchClient.java:254) ~[classes/:?] at org.graylog.storage.opensearch2.OpenSearchClient.execute(OpenSearchClient.java:177) ~[classes/:?] at org.graylog.storage.opensearch2.ClusterAdapterOS2.nodesInfo(ClusterAdapterOS2.java:299) ~[classes/:?] at org.graylog2.telemetry.rest.TelemetryService.getSearchClusterInfo(TelemetryService.java:209) ~[classes/:?] ... Caused by: org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'NodeInfoAction.destructiveRequiresName'

Possible Solution

Per @dennisoelkers this may be due to in progress client migration

Steps to Reproduce (for bugs)

  1. Restart server and login

Your Environment

patrickmann commented 8 months ago

Full stack trace:

2024-03-14 09:13:31,401 ERROR: org.graylog2.shared.rest.exceptionmappers.AnyExceptionClassMapper - Unhandled exception in REST resource org.graylog.shaded.opensearch2.org.opensearch.OpenSearchException: Could not read nodes list from indexer! at org.graylog.storage.opensearch2.OpenSearchClient.exceptionFrom(OpenSearchClient.java:254) ~[classes/:?] at org.graylog.storage.opensearch2.OpenSearchClient.execute(OpenSearchClient.java:177) ~[classes/:?] at org.graylog.storage.opensearch2.ClusterAdapterOS2.nodesInfo(ClusterAdapterOS2.java:299) ~[classes/:?] at org.graylog2.telemetry.rest.TelemetryService.getSearchClusterInfo(TelemetryService.java:209) ~[classes/:?] at org.graylog2.telemetry.rest.TelemetryService.getTelemetryResponse(TelemetryService.java:114) ~[classes/:?] at org.graylog2.telemetry.rest.TelemetryResource.get(TelemetryResource.java:69) ~[classes/:?] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?] at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) ~[?:?] at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.base/java.lang.reflect.Method.invoke(Method.java:568) ~[?:?] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:146) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:189) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:219) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:93) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:478) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:400) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) ~[jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:261) [jersey-server-3.1.5.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [jersey-common-3.1.5.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [jersey-common-3.1.5.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [jersey-common-3.1.5.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [jersey-common-3.1.5.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [jersey-common-3.1.5.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [jersey-common-3.1.5.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:240) [jersey-server-3.1.5.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:697) [jersey-server-3.1.5.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:367) [jersey-container-grizzly2-http-3.1.5.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:190) [grizzly-http-server-4.0.1.jar:4.0.1] at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:212) [metrics-core-4.2.25.jar:4.2.25] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?] at java.base/java.lang.Thread.run(Thread.java:840) [?:?] Caused by: org.opensearch.client.util.MissingRequiredPropertyException: Missing required property 'NodeInfoAction.destructiveRequiresName' at org.opensearch.client.util.ApiTypeHelper.requireNonNull(ApiTypeHelper.java:89) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.opensearch.nodes.info.NodeInfoAction.(NodeInfoAction.java:57) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.opensearch.nodes.info.NodeInfoAction.(NodeInfoAction.java:50) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.opensearch.nodes.info.NodeInfoAction$Builder.build(NodeInfoAction.java:114) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.opensearch.nodes.info.NodeInfoAction$Builder.build(NodeInfoAction.java:94) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:92) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:369) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.JsonpDeserializerBase$StringMapDeserializer.deserialize(JsonpDeserializerBase.java:355) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer$FieldObjectDeserializer.deserialize(ObjectDeserializer.java:81) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:185) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectDeserializer.deserialize(ObjectDeserializer.java:146) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.JsonpDeserializer.deserialize(JsonpDeserializer.java:87) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.ObjectBuilderDeserializer.deserialize(ObjectBuilderDeserializer.java:91) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.json.DelegatingDeserializer$SameType.deserialize(DelegatingDeserializer.java:55) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.transport.rest_client.RestClientTransport.decodeResponse(RestClientTransport.java:317) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.transport.rest_client.RestClientTransport.getHighLevelResponse(RestClientTransport.java:283) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:143) ~[opensearch-java-2.9.0.jar:?] at org.opensearch.client.opensearch.nodes.OpenSearchNodesClient.info(OpenSearchNodesClient.java:145) ~[opensearch-java-2.9.0.jar:?] at org.graylog.storage.opensearch2.ClusterAdapterOS2.lambda$nodesInfo$19(ClusterAdapterOS2.java:299) ~[classes/:?] at org.graylog.storage.opensearch2.OpenSearchClient.execute(OpenSearchClient.java:175) ~[classes/:?] ... 31 more

moesterheld commented 8 months ago

@dennisoelkers This is most likely caused by an incompatibility in OS versions. I had the error in 2.10. I tried setting action.destructive_requires_name by hand, however it was never picked up by the nodes info endpoint. It went away after upgrading to 2.11

Unfortunately, it is expected to be there by the client's NodeInfoAction object image

moesterheld commented 8 months ago

Sorry, I just saw that @patrickmann is already on 2.11 . Can you try to set the property using _cluster/settings to see if it goes away?

patrickmann commented 8 months ago

Can you try to set the property using _cluster/settings to see if it goes away?

@moesterheld When I add that setting I then get the other error you mentioned:

2024-03-14 13:51:38,535 ERROR: org.graylog2.periodical.IndexerClusterCheckerThread - Error while trying to check Elasticsearch disk usage.Details: Cannot invoke "jakarta.json.JsonValue.asJsonObject()" because "current" is null

dennisoelkers commented 8 months ago

This is related to a client bug: https://github.com/opensearch-project/opensearch-java/issues/894

moesterheld commented 8 months ago

@dennisoelkers I will write a note to set action.destructive_requires_name to the 6.0 release notes and add it explicitly in the data node config for OS to avoid problems until this is potentially fixed.

janheise commented 8 months ago

fixed by #18648