ing-bank / scruid

Scala + Druid: Scruid. A library that allows you to compose queries in Scala, and parse the result back into typesafe classes.
Apache License 2.0
115 stars 29 forks source link

Some query context parameters cannot represented by string values #111

Closed anskarl closed 3 years ago

anskarl commented 3 years ago

When setting, for example, a priority parameter value in the query context as a string value (e.g., "1") like below:

val query: TimeSeriesQuery = DQL.timeseries
  .granularity(GranularityType.Hour)
  .interval("2011-06-01/2017-06-01")
  .agg(count as "count")
  .setQueryContextParam(QueryContext.Priority, "1")
  .build()

and in Druid we have query laning enabled, for example with the following settings:

#
# Query laning to test priorities
#
druid.query.scheduler.laning.strategy=hilo
druid.query.scheduler.laning.maxLowPercent=30
druid.query.default.context.lane=low
druid.query.scheduler.prioritization.strategy=manual

The resulting query (with some additional defaults (lane, and query id)) is the following:

{
   "queryType" : "timeseries",
   "granularity" : "HOUR",
   "intervals" : {
      "intervals" : [
         "2011-06-01T00:00:00.000Z/2017-06-01T00:00:00.000Z"
      ],
      "type" : "LegacySegmentSpec"
   },
   "postAggregations" : [],
   "aggregations" : [
      {
         "name" : "count",
         "type" : "count"
      }
   ],
   "limit" : 2147483647,
   "virtualColumns" : [],
   "dataSource" : {
      "name" : "wikipedia",
      "type" : "table"
   },
   "descending" : false,
   "context" : {
      "queryId" : "72afa724-fcac-498f-b5ef-31fe5da69175",
      "priority" : "1",
      "lane" : "low"
   }
}

You will get the following error: The future returned an exception of type: ing.wbaa.druid.client.HttpStatusException, with message: Received response with HTTP status code 500 Internal Server Error.

For more detailed view, in the broker logs you will get the corresponding error:

2021-05-24T06:54:21,145 WARN [qtp856880372-124[timeseries_[wikipedia]_72afa724-fcac-498f-b5ef-31fe5da69175]] org.apache.druid.server.QueryLifecycle - Exception while processing queryId [72afa724-fcac-498f-b5ef-31fe5da69175] (java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer)
2021-05-24T06:54:21,149 ERROR [qtp856880372-124[timeseries_[wikipedia]_72afa724-fcac-498f-b5ef-31fe5da69175]] org.apache.druid.server.QueryResource - Exception handling request: {class=org.apache.druid.server.QueryResource, exceptionType=class java.lang.ClassCastException, exceptionMessage=java.lang.String cannot be cast to java.lang.Integer, query={"queryType":"timeseries","dataSource":{"type":"table","name":"wikipedia"},"intervals":{"type":"LegacySegmentSpec","intervals":["2011-06-01T00:00:00.000Z/2017-06-01T00:00:00.000Z"]},"descending":false,"virtualColumns":[],"filter":null,"granularity":"HOUR","aggregations":[{"type":"count","name":"count"}],"postAggregations":[],"limit":2147483647,"context":{"lane":"low","priority":"-1","queryId":"72afa724-fcac-498f-b5ef-31fe5da69175"}}, peer=172.21.0.1} (java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer)

The error can be fixed by representing the priority value by an integer, for example:

{
   "queryType" : "timeseries",
   "granularity" : "HOUR",
   "intervals" : {
      "intervals" : [
         "2011-06-01T00:00:00.000Z/2017-06-01T00:00:00.000Z"
      ],
      "type" : "LegacySegmentSpec"
   },
   "postAggregations" : [],
   "aggregations" : [
      {
         "name" : "count",
         "type" : "count"
      }
   ],
   "limit" : 2147483647,
   "virtualColumns" : [],
   "dataSource" : {
      "name" : "wikipedia",
      "type" : "table"
   },
   "descending" : false,
   "context" : {
      "queryId": "5f5a1aa1-1e6a-4624-b90b-3acb04c179c5",
      "priority" : 1,
      "lane" : "low"
   }
}