[BUG] Error thrown when getting historical top n queries

[ansjcy]

What is the bug?

Getting this error when parsing historical top n queries data:

[2024-09-06T14:34:57,607][WARN ][r.suppressed             ] [integTest-0] path: /_insights/top_queries, params: {from=2024-09-05T21:34:57.493Z, to=2024-09-06T21:34:57.493Z, type=latency}
java.lang.NullPointerException: Cannot invoke "org.opensearch.plugin.insights.rules.model.Measurement.getMeasurement()" because the return value of "java.util.Map.get(Object)" is null
        at org.opensearch.plugin.insights.rules.model.SearchQueryRecord.getMeasurement(SearchQueryRecord.java:276) ~[?:?]
        at org.opensearch.plugin.insights.rules.model.SearchQueryRecord.compare(SearchQueryRecord.java:378) ~[?:?]
        at org.opensearch.plugin.insights.core.service.TopQueriesService.lambda$getTopQueriesRecordsFromIndex$6(TopQueriesService.java:401) ~[?:?]
        at java.base/java.util.TimSort.countRunAndMakeAscending(TimSort.java:355) ~[?:?]
        at java.base/java.util.TimSort.sort(TimSort.java:220) ~[?:?]
        at java.base/java.util.Arrays.sort(Arrays.java:1308) ~[?:?]
        at java.base/java.util.ArrayList.sort(ArrayList.java:1804) ~[?:?]
        at java.base/java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:392) ~[?:?]
        at java.base/java.util.stream.Sink$ChainedReference.end(Sink.java:261) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
        at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
        at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
        at org.opensearch.plugin.insights.core.service.TopQueriesService.getTopQueriesRecordsFromIndex(TopQueriesService.java:402) ~[?:?]
        at org.opensearch.plugin.insights.rules.transport.top_queries.TransportTopQueriesAction.newResponse(TransportTopQueriesAction.java:95) ~[?:?]
        at org.opensearch.plugin.insights.rules.transport.top_queries.TransportTopQueriesAction.newResponse(TransportTopQueriesAction.java:33) ~[?:?]
        at org.opensearch.action.support.nodes.TransportNodesAction.newResponse(TransportNodesAction.java:179) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction.lambda$finishHim$1(TransportNodesAction.java:321) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:74) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.ActionRunnable$2.doRun(ActionRunnable.java:89) ~[opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.OpenSearchExecutors$DirectExecutorService.execute(OpenSearchExecutors.java:341) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction.finishHim(TransportNodesAction.java:321) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction.onOperation(TransportNodesAction.java:306) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleResponse(TransportNodesAction.java:283) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleResponse(TransportNodesAction.java:275) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$9.handleResponse(TransportService.java:1701) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$ContextRestoreResponseHandler.handleResponse(TransportService.java:1483) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$DirectResponseChannel.processResponse(TransportService.java:1566) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1546) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:72) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.action.support.nodes.TransportNodesAction$NodeTransportHandler.messageReceived(TransportNodesAction.java:334) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.wlm.WorkloadManagementTransportInterceptor$RequestHandler.messageReceived(WorkloadManagementTransportInterceptor.java:63) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:108) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.transport.TransportService$7.doRun(TransportService.java:1036) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:982) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at org.opensearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:52) [opensearch-3.0.0-SNAPSHOT.jar:3.0.0-SNAPSHOT]
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) [?:?]
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) [?:?]
        at java.base/java.lang.Thread.run(Thread.java:1583) [?:?]

How can one reproduce the bug?

• Run the OpenSearch cluster with historical top queries changes https://github.com/opensearch-project/query-insights/pull/84, also run OSD with query insights dashboard.
• Enable top queries by latency, cpu and memory.
• Insert document for searching.
• Make search requests.
• On query insights dashboard, hit refresh, or make an API request:

  curl -X GET "localhost:9200/_insights/top_queries?from=2024-09-05T21%3A34%3A57.493Z&to=2024-09-06T21%3A34%3A57.493Z"
  {"error":{"root_cause":[{"type":"null_pointer_exception","reason":"Cannot invoke \"org.opensearch.plugin.insights.rules.model.Measurement.getMeasurement()\" because the return value of \"java.util.Map.get(Object)\" is null"}],"type":"null_pointer_exception","reason":"Cannot invoke \"org.opensearch.plugin.insights.rules.model.Measurement.getMeasurement()\" because the return value of \"java.util.Map.get(Object)\" is null"},"status":500}%

• Observed the above bug in the backend log

What is the expected behavior?

No error should be thrown.

What is your host/environment?

macOS

[ansjcy]

Will pick this up for 2.18.

[ansjcy]

Note: this only happens when local index exporter is enabled.