search

IGS / portal_client

Python-based client for downloading data made available through portals powered by the GDC-based portal system..
MIT License
17 stars 17 forks source link

Restitute path hierarchy from manifest URLs #14

Open alicedb2 opened 5 years ago

alicedb2 commented 5 years ago

When downloading from a large manifest, all files are dumped in the same directory with no control beyond --destination. When multiple studies and body-sites are included, if would help if the client could save each file in subdirectories that recapitulate part of the URL.

Closest thing I can think of is how wget -r -x -nH --cut-dirs operates, so that the client would end up putting, for example, http://downloads.hmpdacc.org/data/HM16STR/HMDEMO/SRP002429/stool/not_affected/SRS066677.fsa locally in destination-dir/HM16STR/HMDEMO/SRP002429/stool/not_affected/SRS066677.fsa.

victor73 commented 5 years ago

Propose to add a --preserve-dirs flag to add this behavior. The “flat” download behavior would remain the default.