Refactor querying system allowing more flexible querying

[Maelstromeous]

We need to slightly refactor our sorting / querying system to allow inclusion of JSON encoded objects. This should enable us, via some sanity checking, to be extremely flexible in how we allow users to query for data. We'll quite simply decode this string and send the result to mongo as we would normally.

Example:

GET /some/endpoint/1234?filter={instance:"10-37435",facility:{$gt: 7000}}&sort={timeStarted:1}

This will return the result as expected from querying mongo directly.

• [ ] Implement changes to query options parsing the object and then passing that to Mongo
• [ ] Ensure that SQL-eqse injection is not possible via this method.

[microwavekonijn]

That is a proprietary format. A normal format based on the example would be:

GET /some/endpoint/1234?filter[instance]=10-37435&filter[facility][gt]=7000&sort[timeStarted]=1

But if you start looking at such flexibility I would seriously start considering grapQL.

Also considering that most instance collections(except maybe deads) will be around ~1000 records I think sorting can be done locally easily.

[zhenghung]

I'm not sure how I feel about passing in json strings as a query. I've never seen it implemented like that.

[Maelstromeous]

@microwavekonijn

That is a proprietary form

It's not, it's literally what you pass to mongo compass, essentially we're mocking exactly that object literal into the query, parsing it, then passing it to the mongo platform to pull out via the API. We're also declaring against the where calls directly so it presumably won't be open to any kind of injection.

I'm not sure how I feel about passing in json strings as a query. I've never seen it implemented like that.

It'll be up to the client implementation to parse it as a URL encoded string, which I've seen many times, and is pretty safe unless you're using weird things like emoji.

Example:

GET http://dev.api.ps2alerts.com/instances/territory-control?filter=%7B%22world%22%3A17%7D&sort=%7B%22zone%22%3A%201%7D

aka

GET http://dev.api.ps2alerts.com/instances/territory-control?filter={"world":17}&sort={"zone": 1}

Will return a ascending sorted array of territory control alerts from Emerald.

[Maelstromeous]

Which spits out object:

{ where: { world: 17 }, order: { zone: 1 }, limit: 50, skip: null }

Which is then accepted by Mongo and processed using their typehinted object FindManyOptions, fully supporting pagination etc. My intent is we treat it basically how we'd treat any mongo instance.

[microwavekonijn]

I would be very careful with it and have some strict validation in place. But it is still a proprietary form, not as much as how we use it internally but definitely seen from the client server communication. I don't like it, but I won't object.

[Maelstromeous]

Superceded by #64 .