This PR introduces the functionality described in #31. Before we merge it, there are a few things to do:
[ ] Add more tests to ensure that everything is working as expected.
[x] Add a in operator to allow membership checking (which means adding support for [lists, of, items] in the language syntax).
[ ] Add operators for startswith and endswith, as well as matches (glob or regex).
[ ] Make sure that our error messages are actually helpful (maybe port the Location information stuff from Lox to make it easier for users to understand where the error is in a query).
[ ] Give some more comprehensive examples of how to use this stuff.
Background
Currently the filtering model we've implemented is built around serde's ability to deserialize YAML tags (i.e. !Something) into enums and we've paired this with a rudimentary "name + tags" model that works relatively well for GitHub repositories (since we're scoped to a user/org to start with, and the identifying traits of a repo can easily be represented with tags). Doing so looks something like this:
The trouble comes in when we start wanting to support GitHub release artifacts. In this scenario there are several different tiers at which we may want to filter:
The repository that the artifact belongs to.
The release that the artifact is a part of.
The artifact itself.
It's also not entirely obvious what the name should be (for the !Include and !Exclude operators) - right now we've settled for the name of the repo to match semantics with the github/repo model, but it really isn't ideal (as shown below).
What we'd ideally like to be able to do is define a filter that can reference individual properties and perform comparisons. A naive approach would be to model this as follows:
But the issue with this model is that it's difficult to construct combined conditions. What we really, really want is a DSL that lets us construct logical expressions which can then be evaluated by the backup system to determine whether to include something or not. Enter our filter DSL:
Truthiness and falseiness (i.e. release.prerelease) as well as unary negation of this (i.e. !repo.fork).
Binary equality and inequality (i.e. repo.name == "grey" or repo.name != "git-tool").
Some string comparison operations (i.e. repo.name contains "-rs").
Binary logical operations (i.e. A && B or C || D).
Logical expression grouping (i.e. (A && B) || C).
We'd also like it to be reasonably easy to extend with support for things like glob/regex matches, startswith and endswith etc. Non-goals here are the creation of anything Turing complete, impure (touching the environment/performing I/O) or scoped beyond the context of a single entity (i.e. we're not going to let you do joins or match on sibling items).
This PR introduces the functionality described in #31. Before we merge it, there are a few things to do:
inoperator to allow membership checking (which means adding support for[lists, of, items]in the language syntax).startswithandendswith, as well asmatches(glob or regex).Locationinformation stuff from Lox to make it easier for users to understand where the error is in a query).Background
Currently the filtering model we've implemented is built around serde's ability to deserialize YAML tags (i.e.
!Something) intoenums and we've paired this with a rudimentary "name + tags" model that works relatively well for GitHub repositories (since we're scoped to a user/org to start with, and the identifying traits of a repo can easily be represented with tags). Doing so looks something like this:The trouble comes in when we start wanting to support GitHub release artifacts. In this scenario there are several different tiers at which we may want to filter:
It's also not entirely obvious what the
nameshould be (for the!Includeand!Excludeoperators) - right now we've settled for the name of the repo to match semantics with thegithub/repomodel, but it really isn't ideal (as shown below).What we'd ideally like to be able to do is define a filter that can reference individual properties and perform comparisons. A naive approach would be to model this as follows:
But the issue with this model is that it's difficult to construct combined conditions. What we really, really want is a DSL that lets us construct logical expressions which can then be evaluated by the backup system to determine whether to include something or not. Enter our filter DSL:
The goal of this DSL is to cover the following:
release.prerelease) as well as unary negation of this (i.e.!repo.fork).repo.name == "grey"orrepo.name != "git-tool").repo.name contains "-rs").A && BorC || D).(A && B) || C).We'd also like it to be reasonably easy to extend with support for things like glob/regex matches,
startswithandendswithetc. Non-goals here are the creation of anything Turing complete, impure (touching the environment/performing I/O) or scoped beyond the context of a single entity (i.e. we're not going to let you do joins or match on sibling items).