search

czbiohub-sf / MIDAS

Metagenomic Intra-Species Diversity Analysis (MIDAS)
MIT License
35 stars 10 forks source link

Add --no-sign-request flag to 'aws s3 cp' #59

Closed bsmith89 closed 3 years ago

bsmith89 commented 3 years ago

Here's the minimum-viable patch to iggtools that allows me to download with aws s3 cp --only-show-errors --no-sign-request s3://microbiome-igg/<SOME-FILE-PATH>.

When the new flag is excluded I get download failures like the following:

1629844368.0:  'aws s3 cp --only-show-errors s3://microbiome-igg/2.0/pangenomes/100022/centroids.ffn.lz4 - | lz4 -dc > ref/IGGdb2.0/pangenomes/100022/centroi
ds.ffn'
download failed: s3://microbiome-igg/2.0/pangenomes/100022/centroids.ffn.lz4 to - Unable to locate credentials
1629844532.5:  'rm -f ref/IGGdb2.0/pangenomes/100022/centroids.ffn'
1629844533.5:  Sleeping 54.31506110122879 seconds before retry 1 of <function download_reference at 0x7f1062e56d40> with ('s3://microbiome-igg/2.0/pangenomes
/100022/centroids.ffn.lz4', 'ref/IGGdb2.0/pangenomes/100022'), {}.
bsmith89 commented 3 years ago

Just realized this was basically an exact duplicate of #31 ...

I'll leave it open for now, though, because that branch has conflicts with the current HEAD of the master branch.

zhaoc1 commented 3 years ago

Thanks Byron! I haven't got time to tackle AWS related issues. I am gonna merge your changes!