Use Git LFS

[idleberg]

I have noticed that benny has some binary files checked in and would recommend using Git LFS for those. I made the necessary changes locally, but git forbids me to push those changes to my fork.

Chat GPT was so kind to summarize the advantages of using Git LFS better than I could:

When checking large files into Git, several problems can arise:

• Repository Size Growth: Git repositories can grow significantly in size if large files are committed, because Git stores the entire history of changes. Each version of a large file adds to the repository size, making it cumbersome to clone or pull.

• Slow Operations: Operations such as cloning, pulling, and pushing can become very slow with large repositories. This is particularly problematic when every developer has to work with the entire history of large files, even if they only need the latest version.

• Storage Inefficiency: Storing large binary files is inefficient with Git's delta-based storage system, which is optimized for text files. Binary files do not compress well with delta compression, leading to inefficient use of storage.

• Network Bandwidth: Pushing and pulling large files consumes considerable network bandwidth, which can slow down the development workflow and impact other users on the same network.

Git Large File Storage (LFS) addresses these issues by:

• Pointer Files: Git LFS replaces the large files in your repository with small pointer files that reference the actual large files. These pointer files are lightweight and do not significantly increase the size of the repository.

• External Storage: The actual large files are stored outside the Git repository in a separate storage location (e.g., a cloud storage service or a dedicated LFS server). This keeps the repository size manageable.

• Efficient Transfers: When cloning or pulling, only the necessary versions of the large files are downloaded. Git LFS fetches the large files on demand, meaning you only get the files you need for the current checkout, significantly reducing the amount of data transferred.

• Optimized for Large Files: Git LFS is designed to handle large binary files efficiently. It provides better performance for these files compared to Git's built-in storage system.

• Compatibility: Git LFS integrates seamlessly with existing Git workflows, requiring minimal changes to how developers work with repositories.

Instead, I want to share a step-by-step guide on how to achieve this:

0. Install Git LFS

Download the installer or install it with a package manager

# macOS or Linux with Homebrew installed
$ brew install git-lfs

# Debian / Ubuntu
$ sudo apt-get install git-lfs

Activate Git LFS in the repository:

$ git lfs install

1. Track files in Git LFS

Specify which file types or individual files should be managed by Git LFS

git lfs track "*.dll"
git lfs track "*.dmg"
git lfs track "*.so"

This will modify the existing .gitattributes and add new filters. It should look something like this:

# Auto detect text files and perform LF normalization
* text=auto

# Use Git LFS for large, binary files
*.dll filter=lfs diff=lfs merge=lfs -text
*.dmg filter=lfs diff=lfs merge=lfs -text
*.so filter=lfs diff=lfs merge=lfs -text

2. Check in this file

Commit your changes

3. Rewrite History

Next up, it is recommended to add these files to Git LFS retroactively.

git lfs migrate import --include="*.dll"
git lfs migrate import --include="*.dmg"
git lfs migrate import --include="*.so"

:warning: Rewriting history is a significant operation that will change commit hashes. This will require force-pushing the changes to your remote repository and all collaborators will need to re-clone the repository.

4. Notify the team

Contributors should clone the repository anew to get the new history. They also need to install and setup Git LFS as a one-time operation.

---

Again, I'm sorry I could not contribute these changes myself. Also, I understand if you don't want to follow this recommendation for whatever reasons 😉

[jamesholdenmusic]

this is nonsense. go away.

[idleberg]

Not sure what I did wrong, but I didn't intend to offend you. I don't mind you closing the issue, but asking me to "go away" feels… odd?

[jamesholdenmusic]

The request was not in any way helpful or relevant - there are very few binaries here and they're just bundled files that won't change. I assumed it was automated reputation farming. Chatgpt is not welcome here. Hope that clarifies.

[idleberg]

That‘s cool, I get that. Even though only the explanation in the blockquote was from ChatGPT.