search

flatironinstitute / ccn-template

Template repository for CCN software projects
https://ccn-template.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
7 stars 2 forks source link

git-lfs examples #52

Open kushalkolar opened 7 months ago

kushalkolar commented 7 months ago

git-lfs is useful when there are big binary files in the repo that change frequently (such as test data, ground truths, images etc.). Do you have interest in adding that to template? It requires the following:

.gitattributes, example:

https://github.com/fastplotlib/fastplotlib/blob/5053f02582d68c18d0bec5b9b3c4c5371e785139/.gitattributes#L1-L2

Explicit git-lfs install and pull in the CI workflow. If git-lfs is installed before a repo is cloned, then the lfs files are pulled automatically. However if git-lfs is not installed beforehand, which is the case with actions/checkout, then an explicit git lfs pull is required after cloning the repo to get the lfs files.

Example: https://github.com/fastplotlib/fastplotlib/blob/5053f02582d68c18d0bec5b9b3c4c5371e785139/.github/workflows/ci.yml#L88-L91

billbrod commented 7 months ago

I'd be interested in this. I think I'd like to see a discussion of possible alternatives. I already put together #36 , which describes how I've used OSF storage to do something similar, so it's probably worth describing what to consider when deciding between OSF (which does do versioning and allows you to download with a wget or simple python requests), git-lfs, and anything else.