eoscostarica / eos-rate

EOS Block Producer Ratings and Voting Portal :star2:
https://eosrate.io
MIT License
27 stars 27 forks source link

Optimize repo size #241

Closed xavier506 closed 4 years ago

xavier506 commented 4 years ago

Remove large unused git objects to reduce repo size.

The largest file is an idx called something like .git/objects/pack-f8... that has a large object of over 70mb thats making cloninfg the repo really slow.

we need to remove that object from the history también.

rubenabix commented 4 years ago

@danazkari I made a fork and try with https://rtyley.github.io/bfg-repo-cleaner/ and everything is ok but in the eosrate repo I found an error.

! [remote rejected] refs/pull/94/head -> refs/pull/94/head (deny updating a hidden ref)

https://github.com/rtyley/bfg-repo-cleaner/issues/36

Maybe we can try https://help.github.com/en/github/authenticating-to-github/removing-sensitive-data-from-a-repository

rubenabix commented 4 years ago

`➜ eosrate java -jar /Users/rub/Downloads/bfg-1.13.0.jar --strip-blobs-bigger-than 60M eos-rate.git

Using repo : /Users/rub/Documents/git/EOSCR/eosrate/eos-rate.git

This repo has been processed by The BFG before! Will prune repo before proceeding - to avoid unnecessary cleaning work on unused objects... Completed prune of old objects - will now proceed with the main job!

Scanning packfile for large blobs: 11000 Scanning packfile for large blobs completed in 65 ms. Found 1 blob ids for large blobs - biggest=70592168 smallest=70592168 Total size (unpacked)=70592168 Found 178 objects to protect Found 113 commit-pointing refs : HEAD, refs/heads/226, refs/heads/44, ...

Protected commits

These are your protected commits, and so their contents will NOT be altered:

Cleaning

Found 543 commits Cleaning commits: 100% (543/543) Cleaning commits completed in 397 ms.

Updating 69 Refs

Ref                   Before     After   
-----------------------------------------
refs/pull/100/head  | 0e884782 | 3a190fe7
refs/pull/101/head  | d7ffec86 | 5a8e9e15
refs/pull/102/head  | 8c299c6a | 468dc133
refs/pull/106/head  | 7095bdfc | 0ff5809e
refs/pull/109/head  | d6b8d1ac | 715b32d4
refs/pull/112/head  | 6030c71e | e454fb87
refs/pull/113/head  | 3c429ae7 | 0cf1dc41
refs/pull/114/head  | f19d5a96 | 9bcd8478
refs/pull/115/head  | c4e934fe | 532d5427
refs/pull/116/head  | 2ff11c29 | a70704f9
refs/pull/124/head  | 09120c4a | 70966489
refs/pull/131/head  | afddef5d | 0c255b4a
refs/pull/132/head  | e7e039a4 | df8722fb
refs/pull/138/head  | 3b122a8c | 1af96351
refs/pull/143/head  | dd2107db | c8c78112
...

Updating references: 100% (69/69) ...Ref update completed in 70 ms.

Commit Tree-Dirt History

Earliest                                              Latest
|                                                          |
...........DDDDDDDDDDmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm

D = dirty commits (file tree fixed)
m = modified commits (commit message or parents changed)
. = clean commits (no changes to file tree)

                        Before     After   
-------------------------------------------
First modified commit | e053022c | 1c183c62
Last dirty commit     | 35fab144 | 2c2fd08c

Deleted files

Filename                     Git id            
-----------------------------------------------
eosio.cdt-1.4.0.x86_64.deb | e140d558 (67.3 MB)

In total, 357 object ids were changed. Full details are logged here:

/Users/rub/Documents/git/EOSCR/eosrate/eos-rate.git.bfg-report/2020-01-18/11-33-46

BFG run is complete! When ready, run: git reflog expire --expire=now --all && git gc --prune=now --aggressive`

eosio.cdt-1.4.0.x86_64.deb | e140d558 (67.3 MB)

rubenabix commented 4 years ago

I used, the repo size is fixed

git filter-branch --index-filter "git rm --cached --ignore-unmatch *.deb" --tag-name-filter cat -- --all