add: s3 input - Githubissues

nextstrain / dengue

Nextstrain build for dengue virus

https://nextstrain.org/dengue

8 stars 11 forks source link

add: s3 input #5

Closed j23414 closed 2 years ago

j23414 commented 2 years ago

Description of proposed changes

Replace fauna input for s3 input, which will be consistent with:
- nextstrain/zika:Snakefile#L16
- nextstrain/measles:Snakefile#L15
Use zstd compression format since it is faster than either xz and gz, which will be consistent with:
- https://github.com/nextstrain/ncov-ingest/pull/345
- Update example data to zst format for nextstrain/.github/workflows...pathogen-repo-ci.yaml#L133

Related issue(s)

Testing

[x] Checks pass

Assuming the reviewer has the proper keys, could run this manually via:

git clone https://github.com/nextstrain/dengue.git
cd dengue
git checkout s3
nextstrain build .

And make sure entire pipeline builds, or check the data folders for pulled zst files.

ls -1 data 
metadata_all.tsv.zst
metadata_denv1.tsv.zst
metadata_denv2.tsv.zst
metadata_denv3.tsv.zst
metadata_denv4.tsv.zst
sequences_all.fasta.zst
sequences_denv1.fasta.zst
sequences_denv2.fasta.zst
sequences_denv3.fasta.zst
sequences_denv4.fasta.zst