search

nextstrain / ncov-ingest

A pipeline that ingests SARS-CoV-2 (i.e. nCoV) data from GISAID and Genbank, transforms it, stores it on S3, and triggers Nextstrain nCoV rebuilds.
MIT License
36 stars 20 forks source link

Use built-in Snakemake `retries` directive #429

Closed joverlee521 closed 8 months ago

joverlee521 commented 8 months ago

Description of proposed changes

We've been using a custom automatic retry for rules that fail occasionally during fetching of online resources since the workflow was still a bash script.¹

When the workflow was converted a Snakemake workflow, Snakemake did not have rule specific retries yet.²

The latest GISAID ingest failed³ due to an error in run_shell_command_n_times, so I figured it's time to loop back and replace it with the built-it Snakemake retries directive that has been available since v7.7.0.

¹ https://github.com/nextstrain/ncov-ingest/commit/75ee3921137f879a94d6996f73770a77727408af ² https://github.com/nextstrain/ncov-ingest/pull/231/files#r758769573 ³ https://bedfordlab.slack.com/archives/CTZKJC7PZ/p1706908574251489?thread_ts=1706905060.565059&cid=CTZKJC7PZ

Related issue(s)

Checklist

joverlee521 commented 8 months ago

Both test builds completed successfully. Merging and will monitor automated runs the next couple of days.