Dealing with large files

analyticalworkflows / TeachingMaterials

Teaching materials for Analytical Workflows, a graduate course at Oregon State University

MIT License

7 stars 19 forks source link

Dealing with large files #43

Open marknovak opened 2 years ago

marknovak commented 2 years ago

If you can't break up your files into smaller chunks and (wisely) prefer to not store them outside your repository ...

Git LFS (Large File Storage) allows you to store 1 GB for free
how-large-does-a-large-file-have-to-be-to-benefit-from-git-lfs
GitHub details on managing-large-files
Uploading-large-files-to-github summary from Towards Data Science, including link of BFG Repo-cleaner for removing large files
More general tutorial-removing-large-files-from-git from Medium

marknovak commented 2 years ago

We should survey everyone at the start of class to get a better sense of what types of large data they are dealing with.

marknovak commented 2 years ago

This topic is brought up (but not discussed or detailed) in "Git Part 2". I think we should elaborate on it there (in conjunction with rethinking the schedule of classes #37).