square / metrics

Metrics Query Engine
Apache License 2.0
170 stars 21 forks source link

Add JSON API for structured queries #254

Closed Nathan-Fenner closed 8 years ago

Nathan-Fenner commented 8 years ago

JSON API for /query path

When a request is made with the header Content-Type: application/json then the server will assume that values are being passed in as a JSON object in the body of the request, rather than as URL parameters. For example, posting the following

{
    "query": "describe all",
    "profile": true
}
{
    "query": "select cpu.percent from -40m to now where host match 'mqe'"
}

The main useful addition is that you can use JSON to describe extra constraints to be placed on the query results (a programmatic where clause):

{
    "query": "select cpu.percent from -40m to now where host match 'mqe'",
    "where": {
        all: [
            {"key_match": {"key": "app", "regex": "foo|qu?x\\("}},
            {"key_in": {"key": "host", "values": ["host5", "host17", "host29"]}}
        ]
    }
}

These predicates mirror the built-in predicates in the where clause. If your query also contains a where clause, then they both must be satisfied for a timeseries to be included.

The full specification for predicates follows:

A predicate is one of the following:

{ "not": P }
{ "any": [P1, P2, ..., Pn] }
{ "all": [P1, P2, ..., Pn] }
{ "key_is": {"key": name, "value": value} }
{ "key_in": {"key": name, "values": [value1, value2, ..., valuen]} }
{ "key_match": {"key": name, "regex": regex} }

JSON Tag-Based Ingestion API through /ingest path

In addition, a JSON API is available that allows you to ingest a metrics of the form

[{"name": "cpu", "tags": {"host": "server12", "app": "mqe"}}]

posted to the body of the request (with Content-Type: application/json).

TODO: add configuration to enable/disable this

Misc Changes

In order to make this change smoother, some other things were moved around (predicate now gets its own package inside query, for example).

Note: the profiling JSON response now consists of timestamp strings instead of millisecond epochs.

@drcapulet @achow @syamp

drcapulet commented 8 years ago

We have any tests covering the new JSON Query API?

Nathan-Fenner commented 8 years ago

There aren't any tests covering the new behavior, I'll add some.

drcapulet commented 8 years ago

:+1:

Might be best to squash all of these commits?

syamp commented 8 years ago

Yes please, for squashing..