search

PanosK92 / SpartanEngine

A game engine with an emphasis on real-time cutting-edge solutions
https://panoskarabelas.com
MIT License
2.24k stars 189 forks source link

Git: Clean up .git folder to reduce repo size #150

Open steyou opened 5 months ago

steyou commented 5 months ago

Greetings,

1. Background

I was curious about this project because I was doing research for my own similar project. I wanted to grep for a particular vulkan API call which meant cloning the repo.

2. The issue

The repo is huge! Almost 8GB.

I was curious about what could possibly balloon the size and found two reasons:

2.1 .git folder

running ncdu reveals that your .git folder is unusually large:

--- /home/steven/desktop/SpartanEngine ------------------
    7.7 GiB [##############] /.git
   42.9 MiB [              ] /third_party
   ...

Looking further reveals that .git/objects is the culprit:

SpartanEngine $ ls -lh .git/objects/pack/
total 7.7G
-r--r--r-- 1 steven steven 2.5M May 30 16:31 pack-5bd27b0d80c7e51945de09f9483027fc1c652adb.idx
-r--r--r-- 1 steven steven 7.7G May 30 16:31 pack-5bd27b0d80c7e51945de09f9483027fc1c652adb.pack
-r--r--r-- 1 steven steven 301K May 30 16:31 pack-5bd27b0d80c7e51945de09f9483027fc1c652adb.rev

A single .pack file which is 7.7G in size.

2.2 Use of binary .7z files

Searching your git history shows extensive use of .7z files to manually compress assets. Below are the top two largest files in your history.

SpartanEngine $ \
git rev-list --objects --all |
git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' |
sort -nk3 |
tail -n 2
blob 342f2af19208bb44343f7eb9bdd85d57a4e04e21 102377268 ThirdParty/libraries/libraries.7z
blob eab0edc4f37003509662cbef92afc0812401da16 102982473 third_party/libraries/libraries.7z

While I see that you've actually removed third_party/libraries folder containing the .7z file(s), they're still an artifact on the repo size. This makes cloning the repo, honestly, a nightmare.

3. Suggestion

I'm curious if leaving the .git/objects folder (.pack file) unkempt is a conscious decision. If not, I believe it's possible to remove that file from the git history. While it does rewrite history and make going back in time much more difficult, I believe the advantage of having a significantly smaller repo outweighs that.

kristiker commented 4 months ago

git clone --depth 1