Add independent ingest and phylogenetic GH Action workflows

nextstrain / zika

Nextstrain build for Zika virus

https://nextstrain.org/zika

8 stars 10 forks source link

Add independent ingest and phylogenetic GH Action workflows #55

Closed joverlee521 closed 2 months ago

joverlee521 commented 2 months ago

Description of proposed changes

Splits the ingest-to-phylogenetic GH Action workflow added in #52 into independent reusable workflows. The reusable GH Action workflows ingest and phylogenetic are then called from within ingest-to-phylogenetic, but they can also be run independently and with trial names.

This splits the use of the workflows as

ingest-to-phylogenetic.yaml for scheduled, automated runs
ingest.yaml for manual trial runs of the ingest workflow
phylogenetic.yaml for manual trial runs of the phylogenetic workflow OR forced runs that don't check the cache

Checklist

[x] Checks pass
[x] Test run of ingest-to-phylogenetic (with cache)
[x] Test run of ingest-to-phylogenetic (after manually deleting cache)
[x] Test run of ingest (with trial name)
[x] Test run of phylogenetic (with trial name and custom input URLs that point to output files of the trial ingest run)

joverlee521 commented 2 months ago

The trial ingest outputs are available at s3://nextstrain-data/files/workflows/zika/trials/reusable-workflows/.

The trial phylogenetic bulid that used the trial ingest output was deployed to staging, available at https://nextstrain.org/staging/zika/trials/reusable-workflows/zika

joverlee521 commented 2 months ago

Merging to allow @j23414 and @kimandrews to see the GitHub Actions UI for the 3 different workflows. If things look good, I'll port these changes to dengue and measles repos as well. Then I'll continue on the GitHub Action workflow templates in https://github.com/nextstrain/.github/tree/pathogen-repo-workflow-templates so we can use them for future pathogen repos.

Post-merge feedback is always welcome!

j23414 commented 2 months ago

Worked as expected during my tests (ingest, phylogenetic). Thanks!

joverlee521 commented 2 months ago

Oh hmm, just found a side effect of this PR. The Pathogen status page no longer includes the automated workflow:

Screenshot 2024-04-22 at 2 14 55 PM

tsibley commented 2 months ago

@joverlee521 The criteria used for that status page, "workflows which use nextstrain/.github/.github/workflows/pathogen-repo-build.yaml", can get tweaked. It was a convenient proxy for "workflows we want to monitor on this page" but we can always create/use a different way to mark/flag/filter workflows for that page.

joverlee521 commented 2 months ago

@joverlee521 The criteria used for that status page, "workflows which use nextstrain/.github/.github/workflows/pathogen-repo-build.yaml", can get tweaked. It was a convenient proxy for "workflows we want to monitor on this page" but we can always create/use a different way to mark/flag/filter workflows for that page.

For sure! Just need to figure out what the appropriate proxy is for the workflows of this PR. I'll write up something in the status repo.

kimandrews commented 2 months ago

I completed trial runs and it went fine for me too: ingest, phylogenetic