khanlab / hippunfold

BIDS App for Hippunfold (automated hippocampal unfolding and subfield segmentation)
https://hippunfold.readthedocs.io
MIT License
47 stars 12 forks source link

Download resources from osf #277

Closed akhanf closed 4 months ago

akhanf commented 4 months ago

Adds a download.smk rules file and a couple of rules to download+extract zip files from osf for templates and atlases. Will go in template and atlas subfolders in the hippunfold cache dir (ie HIPPUNFOLD_CACHE_DIR, or appdir's cache which defaults to the home directory).

The files are extracted using a snakemake output directory(), so the contents are not monitored by snakemake, only the folders. So for downstream rules the files have to be specified in params, using the folder (ie atlas_dir, template_dir) in the inputs. Only downside is a dry-run without files downloaded won't be able to check to see if the config['template_files'] or config['atlas_files'] entries are correct. But could just do hippunfold --until download_extract_atlas_or_template -c 1 to pre-download the relevant resources, then a dry-run after that would work..

Note: the folder structure of the HIPPUNFOLD_CACHE_DIR will change since models won't be in the top level. We could document this perhaps in the next release notes to indicate how to move them:

mkdir $HIPPUNFOLD_CACHE_DIR/model && mv -v $HIPPUNFOLD_CACHE_DIR/*.model_best.tar $HIPPUNFOLD_CACHE_DIR/model
akhanf commented 4 months ago

Just realized I forgot the upenn template, will do that now

akhanf commented 4 months ago

actually, I'll leave tpl-upenn there for now, as the shape-inject stuff gets edited with the template-based segmentation edits which will be in another PR..

jordandekraker commented 4 months ago

Very nice!

Perhaps this would be a good branch to scrub the large files (i.e. mostly resources) from our git history to reduce the size of clones?

akhanf commented 4 months ago

I think that would be done at the git repository level since it would scrub things beyond this branch.. but I'll have a look now