search

spotify / heroic

The Heroic Time Series Database
https://spotify.github.io/heroic/
Apache License 2.0
847 stars 109 forks source link

RFC: Query Language #481

Open hexedpackets opened 5 years ago

hexedpackets commented 5 years ago

Problem

Querying Heroic with JSON can be verbose and hard to manually do. Heroic currently has its own query language, HQL, but we haven't kept it up to date with changes to the codebase.

Suggestion

We should compare this to other syntaxes, like PromQL. If possible it would be better to support something more standard rather than continuing to roll our own. If that's not possible we need to update HQL and provide support for it in our query sources (particularly the Grafana plugin).

We also need to consider how much benefit we get from supporting a query language at all. JSON may be good enough.

hexedpackets commented 5 years ago

As points of comparison, OpenTSDB has its own query language. InfluxDB has a SQL-like query language and supports reads through a specific endpoint.

There are actually only a handful of TSDBs that support remote reads from Prometheus.

That said, I don't think it makes sense to support two query languages. If we want to support reading from Prometheus, I would lean towards going all-in on PromQL.

dmichel1 commented 5 years ago

Does going all in PromQL mean adding it within Heroic itself or writing a remote read prometheus adapter that would translate between PromQL and the heroic json API?

The latter approach seems like it would be less upfront work to get going and would give us some benefits our users have been asking for like basic arithmetic without having to implement any of those features. In this case heroic is just returning the raw or sampled time series and letting prometheus do the heavy lifting.

hexedpackets commented 5 years ago

For the adapter, we still have to translate the PromQL query into something Heroic understands, right?

I just don't think we should support multiple DSLs long term. If the experimentation with PromQL looks solid, we should completely replace HQL with it. Having support for multiple DSLs, regardless of whether that's in an external adapter or the core of Heroic, means more work every time we want to add functionality (like new filters).

dmichel1 commented 5 years ago

Is the JSON API considered a DSL?

I completely agree with replacing HQL with PromQL if it looks good. The challenge here might be maintaining this in Java when Prom is written in go?