RC-v1.1.0

[patmagee]

Motivation

This release focuses on two main themes: Observability and Scalability.

Observability: As more people have adopted WES it has become clear that the current API does not provide enough information for end users. The RunStatus and Log models have limited required keys and making actionable decisions on the information that is available is difficult without performing additional requests.

Scalability: As more implementors are looking to "productize" WES, the topic of scalability has continuously come up. Specific features of WES (such as embedded task logs) make implementing WES API a challenge, and can cause the service to break (Ie too many task cause a service to consume too much memory).

What's Changed

• feat: added run state PREEMPTED by @vsmalladi in #188
• feat: add in support for workflow engine and version when submitting a request by @vsmalladi in #182
• feat: add alternative schema for array items in RunListResponse by @uniqueg in #207
  • DEPRECATION NOTICE: the RunListResponse model will only return RunSummary starting in 2.0.0
• feat: created a new endpoint to paginate run tasks by @patmagee in #177
  • DEPRECATION NOTICE: the task_logs property in the RunLog model has been deprecated in favour of the task list endpoint, and will be removed in 2.0.0
• feat: make Tasks Addressable by ID and extend available information in the task log by @patmagee in #205

Additional Improvements

• Removed travis CI integration and added GitHub actions for publishing docs and linting openapi.yaml
• Removed example python client code in b5f782e
• Removed duplicated swagger specification in favour of consolidated OpenAPI3 specification

[patmagee]

@briandoconnor @dglazer this rc has been open for a while. What are the next steps to getting this ratified as versions 1.1 of Wes and released?

[uniqueg]

I may be wrong here, so would indeed be good to have confirmation from @dglazer and/or @briandoconnor.

However, I do think that if these changes have been shared with the community and the community has been given ample opportunity to comment, raise concerns, veto etc., and if that is all fine, I think you should be good to go. As far as I know, minor updates do not require SC approval and no particular paperwork.

So I'd say just tag the head commit with the new version, push the tag and create a release through the GitHub interface.

[dglazer]

Can you confirm this is a backwards-compatible change, specifically meaning that old clients will work fine with new servers? (The other way around is also important, but as long as new clients can check the server version and fall back appropriately it's manageable.) If so, I mostly agree with @uniqueg that no extra process is needed.

Re "shared with the community" -- I agree this has been out there for a while. @briandoconnor , do you think that's sufficient, or should we do one more broad "release coming in two weeks" email? (I'm 70/30 in favor of just doing it, given previous communication.)

[uniqueg]

Without a comprehensive compliance test suite for clients, it's virtually impossible to guarantee that any old client would still work.

For example, consider a response model where additionalProperties is set to True (which is generally the case for most, if not all Cloud API response models; it's the default) but a client falsely sets it to False for validation of that model. Adding a new property (not a breaking change) would break that client, although that client worked just fine before and seemed compliant. This is the client's fault and shouldn't stop us from adding properties.

In terms of checks, I think WES now uses, in its CI, a library that checks the API changes for breaking changes automatically. In addition, I believe that all PRs have been reviewed carefully to ensure backwards compatibility in both directions in theory.

I do think we should work towards better automated testing though. Test suites for servers are now available for DRS and TES, and are in the making for WES. However, these are still incomplete (compliance tests are +/- okay, but functional tests are largely still missing). Test suites for clients, as far as I know, are not available (or even under development?) for any of the Cloud APIs.

[patmagee]

@dglazer as far as I am aware, none of the changes introduced are breaking and are fully backwards compatible. As @uniqueg pointed out, this is functionally hard to test without a complete compliance test suite.

Once @briandoconnor weighs in here, I think we are good to mint the release! @dglazer who is responsible for publishing releases on Zenodo?