ga4gh / workflow-execution-service-schemas

The WES API is a standard way to run and manage portable workflows.
Apache License 2.0
82 stars 38 forks source link

Add simple run filtering on tag values #213

Open patmagee opened 9 months ago

patmagee commented 9 months ago

Currently there is no mechanism to filter runs in WES. This has been an intentional choice to ensure we are not providing an overly burdensome API to implementors. At this point there have been several common use cases which have emerged and already implemented in the community (via TES).

This PR Implements the same filtering strategy implemented by TES as part of this PR. It does not intend to be exhaustive but give a starting point to filtering workflow runs

patmagee commented 9 months ago

@uniqueg FWIW we have implemented tag filtering at DNAstack on our Wes api and have taken a slightly different approach which I think is more natural. I just suggested the current approach since it seemed that was the way tes went, but if there is no consensus on whether it is working or not we can try something else.

Basically you can specify the tag query param as many times as you want, representing a logical and between all of the tags.

The filter has the form tag=(?<key>[\:\&]+)(:(?<value>.+))?

Which then can be used like the following

/runs?tag=foo:bar&tag=bizz&tag=sample:3320

Overall we find this pretty flexible. The use case of filtering by only tag value is really not there imo, and in practice we rarely ever see someone using just a key for filtering (although not does happen).

uniqueg commented 9 months ago

Thanks @patmagee. I like the approach you describe, but with all other things being mostly equal/equivalent, I like consistency even more :)

Less mental strain for users, devs and our future selves including similar provisions in other specs.

It's just that I never tried the TES way of implementing tag filtering myself. And so I think Kyle's and Venkat's feedback may be more valuable here.

But good to have that second option, let's see what others think.

patmagee commented 9 months ago

@uniqueg the main problem I can see with the current approach is that there is an implicit relationship between the order of tag_key and tag_value query params.

By splitting them up you force users to think about the order that filters are supplied in, since it now matters especially when the number of tag values or keys do not match.

patmagee commented 6 months ago

@kellrott @vsmalladi pinging you here again to see if you have any experience with how the TES filtering implementation has been received.