search

AlexsLemonade / OpenScPCA-analysis

An open, collaborative project to analyze data from the Single-cell Pediatric Cancer Atlas (ScPCA) Portal
Other
9 stars 17 forks source link

Add download option to sync script #751

Closed jashapiro closed 2 months ago

jashapiro commented 2 months ago

closes https://github.com/AlexsLemonade/OpenScPCA-admin/issues/257

Here I added a --download flag to the existing sync-results.py script. At the moment, this option works quite simply to reverse the source and destination for the internal aws sync command, and should allow downloading from a results bucket with code like the following, (assuming the sso login has been performed): 

python scripts/sync-results.py cell-type-wilms-tumor-06  --bucket researcher-008971640512-us-east-2 --download

All of the other options continue to work, so to use a different profile from the currently active one and perform a dry run, you would add options as follows:

python scripts/sync-results.py cell-type-wilms-tumor-06  --bucket researcher-008971640512-us-east-2 --download --profile workload --dryrun

My only other thought here is that it might be nice to have the bucket autopopulate for each module if we had a file somewhere that stored the bucket name, such as <module>/.aws_bucket which could save a bit of repeated manual checking. But that could be a later PR.

jashapiro commented 2 months ago

Seems to be working as expected in my local testing 👍🏻

Great! I was hoping this would do everything you needed, but I wasn't 100% sure if there might be any other expectations.