search

templateflow / python-client

A python client to query TemplateFlow via pyBIDS
https://templateflow.org/python-client/
Apache License 2.0
8 stars 12 forks source link

[ENH] Datalad-free alternative for TemplateFlow #7

Closed oesteban closed 5 years ago

oesteban commented 5 years ago

This PR provides an alternative for systems without datalad to pull data from TemplateFlow, using lazy loading.

A new skeleton file containing metadata and empty-files replacing datalad symlinks is now found at templateflow/conf/templateflow-skel.zip. If the repository has not been initiated and datalad is not installed, then this skeleton is unzipped.

When the user requests a file via pybids, the file size is checked. If it is zero (i.e., not a symlink and no content), the file is pulled from an S3 bucket (lazy loading).

The S3 bucket is easily updated with git annex export.

With every build the skeleton file is updated (based off the datalad repository pulled down in a previous step of the build) and added as an artifact to the CircleCI build. If the commit is a tag commit, then the skeleton file is updated before releasing in the deploy_pypi build.

oesteban commented 5 years ago

I'll go ahead and check whether the deployment build also works as expected.