meln5674 / grafana-mongodb-community-plugin

Open Source Grafana Plugin for querying MongoDB
GNU Affero General Public License v3.0
137 stars 18 forks source link

Query writer should support BSON objects #17

Open KristapsT opened 1 year ago

KristapsT commented 1 year ago

The aggregation query to MongoDB should allow it to be written in BSON format.

This is because MongoDB stores values as in BSON format and allowing to write the query in BSON would simplify querying when using BSON fields, for example BSON UUID object.

Example MongoDB document:

{
    "timestamp": "2019-23-31T23:00:00+00:00"
    "metadata": {
        "my_uuid": UUID("5aff47d7-a7b6-4974-b86f-d061623e102b")
    }
    "value": 5
}

Currently to query documents matching that UUID, query must look something like this:

[{
    "$match": {
        "metadata.my_uuid": {
            "$binary": {
                "base64": "Wv9H16e2SXS4b9BhYj4QKw==",
                "subType": "04"
            }
        }
    }
}]

But with BSON support in query it would be:

[{
    "$match": {
        "metadata.my_uuid": UUID("5aff47d7-a7b6-4974-b86f-d061623e102b")
    }
}]
meln5674 commented 1 year ago

UUID(), and the other functions provided by mongosh you are referring to as "BSON", are in fact, JavaScript. The "$binary": { ... } is, in fact, the real "BSON" (or, rather, the equivalent JSON representation) that the UUID() function returns. While I agree that being able to use functions like UUID() in the query editor would be far more convenient, the best way I've found so far to do that would be to embed an entire JavaScript runtime into the plugin backend, which is not something I'd do lightly, both for performance and security reasons.

So, unfortunately, this is not something that can easily be solved in the near future.

If this is something you'd like to contribute, or even just research better ways to accomplish, I'd be more than happy to accept a PR.

lpnc commented 1 year ago

Same question for ObjectId(). Is there a way to match an id from another variable such as ObjectId("${myVariable}")?

meln5674 commented 1 year ago

Same issue, unfortunately. All of the JavaScript functions return extended JSON, so you can use that page to determine how to express your query directly in JSON.

For context, this plugin was originally developed to deal with timeseries collections, where individual objects are sort of meaningless, so I'm only just now going back and making sure it works well with other use cases.

However, it has occurred to me that I don't actually need a full JavaScript interpreter to do this, I just need to be able to parse it, and then walk the syntax tree to produce a valid query object, so I may be able to implement this sooner than I originally expected, but no promises as to a timeline.