FrankChen021 / bithon

An observability platform mainly for Java
Apache License 2.0
14 stars 4 forks source link

Simplified query language #631

Open FrankChen021 opened 10 months ago

FrankChen021 commented 10 months ago
{
        "dataSource": "exception-metrics",
        "type": "timeseries",
        "fields": [
          "eventCount"
        ],
        "bucketCount": 30
      }

It's too verbose. Can be simplied by one expression such as:

-- time serieis
from exception-metrics where appName = 'test' select eventCount group by timestamp
from exception-metrics select eventCount group by timestamp(30)
from exception-metrics select eventCount group by timestamp(30), exceptionClass
from exception-metrics where eventCount group by timestamp(30), exceptionClass

-- groupBy
from exception-metrics select exceptionClass, eventCount by exceptionClass order by exceptionClass asc limit 10

--list
from exception-metrics select timestamp, instanceName, thread, message order by timestamp asc

There other two alternatives or references:

  1. https://prql-lang.org/
  2. PromSQL
FrankChen021 commented 10 months ago

for timeseries,

group by interval(timestamp)
group by interval(timestamp, 30) -- field = timestamp, bucke count = 30
group by interval(timestamp, 30, 60) -- field = timestamp, bucket count = 30, minimal length is 60 seconds

for group by, the grouped field can be omitted in the SELECT clause

from exception-metrics select eventCount group by exceptionClass order by exceptionClass
FrankChen021 commented 10 months ago

The filter can be in the last of the statement so that it simplifies the processing of user input

from exception-metrics select eventCount group by exceptionClass order by exceptionClass WHERE appName = 'test'
FrankChen021 commented 10 months ago

Other things to do

  1. ResultFormat supports columar format
  2. The response of timeseries data follows the general response format to be more simple and flexible
  3. the zero-fil can be optional at the client side
FrankChen021 commented 10 months ago

To simply the processing at the FE, the server can turn this expression into original JSON object format.

FrankChen021 commented 4 months ago

https://promlabs.com/promql-cheat-sheet/

FrankChen021 commented 4 months ago

Extension:

  1. percentage
  2. size
  3. duration resolution Now we support auto resolution and flexible resolution by given fixed bucket count. rate(http_requests_total[5m])[30m:1m]