search

nextstrain / augur

Pipeline components for real-time phylodynamic analysis
https://docs.nextstrain.org/projects/augur/
GNU Affero General Public License v3.0
268 stars 128 forks source link

Release workflow doesn't run tests successfully but continues anyway #1633

Closed tsibley closed 2 months ago

tsibley commented 2 months ago

The test step:

https://github.com/nextstrain/augur/blob/227f52681a5bbd183a7a4993996fcf54dfdb5a57/.github/workflows/release.yaml#L28

doesn't error on test failures because devel/test masks the exit status:

https://github.com/nextstrain/augur/blob/227f52681a5bbd183a7a4993996fcf54dfdb5a57/devel/test#L28-L30

I noticed this with the 26.0.0 release just now, but it's been happening with past release workflow runs too.

While we can "fix" this by propagating the exit status in devel/test, I don't think devel/test is really appropriate for CI/CD, at least in its current developer-focused form. Our release process should ideally use a very similar test setup as CI does, or even be part of the CI/CD workflow.

victorlin commented 2 months ago

How about something like #1634?

tsibley commented 2 months ago

Left commentary there.