nerc-project / nerc-docs

User-facing documentation for the New England Research Cloud
https://nerc-project.github.io/nerc-docs
10 stars 11 forks source link

Add tutorial on using OpenStack Swift for Datalad #125

Open jennydaman opened 1 year ago

jennydaman commented 1 year ago

Hello,

I am using NERC OpenStack Swift as a git-annex special remote for a Datalad repository. I would like to contribute some documentation on how to do this to the NERC user guides.

Background info: Datalad is a git-based data management tool which wraps git-annex, a system for adding large files to Git and using distributed storage for those files. Datalad is very popular among the neuroimaging research community.

I am successfully using NERC Swift as a git-annex special remote for my Datalad repository here: https://github.com/jennydaman/bigbrain-arterial-registration-datalad

Here's what I did:

First, I created EC2 access key ID and secret from https://stack.nerc.mghpcc.org/dashboard/project/api_access/

export AWS_ACCESS_KEY_ID=xxxxxxxx
export AWS_SECRET_ACCESS_KEY=xxxxxxxx
git annex initremote nerc-swift type=S3 encryption=none bucket=bigbrain-arterial-registration-datalad autoenable=true host=stack.nerc.mghpcc.org protocol=https port=13808 requeststyle=path public=yes
git annex enableremote nerc-swift publicurl="https://stack.nerc.mghpcc.org:13808/swift/v1/AUTH_2dd3b02b267242d9b28f94a512ea9ede/bigbrain-arterial-registration-datalad"
datalad create-sibling-github bigbrain-arterial-registration-datalad-3_fsl_registration --access-protocol ssh --publish-depends nerc-swift
computate commented 1 year ago

We appreciate the documentation you wish to share @jennydaman .

@joachimweyl is the best place for this documentation here (https://github.com/ocp-on-NERC/docs), or is there another repo for OpenStack + OpenShift User docs? Who is the right person to assign to documentation issues?

joachimweyl commented 1 year ago

As this is OpenStack-based I believe it would end up in nerc-project not ocp-on-nerc which is OpenShift-based. @Milstein where would you like this (https://github.com/nerc-project/nerc-docs) or somewhere else?