Consider using existing standards for query params (filtering, paging, ordering)

[ben-ryder]

Describe your improvement Consider refactoring the query parameters used for listing endpoints to standadise their usage and allow for more advanced query options.

Is your suggestion related to any problems?

• Filter query params are currently custom for each endpoint. They are easy to use, but going forward make the API less consistent and "industry standard" and will be harder to scale.
• The current filter funcitonality does not allow you to do advanced querying like custom and/or groups, multiple filters per field, negating etc. While the use cases for custom filtering in this specific API are not that important due to most content being encrypted on the client, it could still be useful to standardise.

Additional context

• Consider using a format like Drupal JSON API filtering.
• This will likely related to #28 which is about looking at API standards/specs to follow for the whole API.

[ben-ryder]

listing-param-spec-0.1.md

A v0.1 spec for filtering, pagination and ordering. The filtering is inspired by the Drupal JSON:API filtering params however it is using a nested array structure rather than named groups to reduce the parsing and validation complexity. I'm not 100% happy with this spec, and while the format in JSON is ok the resulting query paramters with indexed arrays is quite cumbersome. It might be worth experimenting with using the same named conditions/queries format as Drupal to see if that can actually be parsed/validated ok, or search for any other existing implementations/specs I could follow.

[ben-ryder]

Reviewing Directus and Strapi they basically use the same approach as each other, with slightly different syntax.

Filter conditions are defined in a key/value object as field: conditions where conditions is another key/value object in the format operator: value, for example:

?filters[field][operator]=value

{
  "filters": {
    "field": {
      "operator": "value"
    }   
  }
}

More advanced grouping of filter conditions can then be achived by defining an object with $or: [] or $and: [] like so:

?filters[$and][0][status][$equal]=published&filters[$and][1][$or][0][name][$in]=testing&filters[$and][1][$or][1][name][$not_equal]=testing1

{
  "filters": {
    "$and": [
      {
        "status": {
          "$equal": "published"
        }
      },
      {
        "$or": [
          {
            "name": {
              "$in": "testing"
            }
          },
          {
            "status": {
              "$not_equal": "testing1"
            }
          }
        ]
      }
    ]
  }
}

I think this does create a nicer query format (especially for query parameters) and it even lets you concisely define multiple conditions for the same field at the same level, for example:

?filters[name][$in]=testing&filters[name][$not_equal]=testing1

{
  "filters": {
    "name": {
      "$equal": "testing",
      "$not_equal": "testing1"
    }   
  }
}

My asusmption is you would treat these as two seperate conditions combined using one "$and" group.
It's like the complexity of the required query format increases proportionally to your query complexity, you can start creating more advanced queires like this without needing to immediately jump into the extra complexity of $and/$or groups.

I like the idea of the character prefix that identifies "system" fields and I prefer Strapi's format of $key over the Directus format of _key as I've seen the convension of using _variable to denote a private variable in JS before.
From looking at various CMS systems, I can also see the https://www.npmjs.com/package/qs library being recommended a lot for generating/parsing query strings. I think I should consider that over https://www.npmjs.com/package/query-string becuase of this.

[ben-ryder]

https://github.com/headbase-app/headbase/blob/main/docs/localful/server/listing-params.md

v1 implementation docs now in repo, using same format as Strapi/Directus (and others) as explained above