search

Planetable / Planet

Build and host decentralized blogs and websites on your Mac
https://planetable.xyz
MIT License
1.45k stars 60 forks source link

Use git-filter-repo to strip historical large IPFS binaries #367

Open livid opened 5 months ago

livid commented 5 months ago

Result of git filter-repo --analyze from 73a7e66acd0b7912ba8ff8317926107aaef1705f:

=== All paths by reverse accumulated size ===
Format: unpacked size, packed size, date deleted, path name
   239504981  108685405 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-amd64
   238619056  104103282 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-arm64
   110305152   39602143 2022-10-11 Planet/Assets.xcassets/IPFS-GO.dataset/ipfs
   110437216   38533437 2022-10-11 Planet/Assets.xcassets/IPFS-GO-ARM.dataset/ipfs
    67460213   33836608 <present>  Planet/IPFS/ipfs-executables/ipfs-amd64-0.28.bin
    67460213   33836608 <present>  Planet/IPFS/go-ipfs-executables/ipfs-amd64-0.28.bin
    65355701   31854693 <present>  Planet/IPFS/ipfs-executables/ipfs-arm64-0.28.bin
    65355701   31854693 <present>  Planet/IPFS/go-ipfs-executables/ipfs-arm64-0.28.bin
    62964533   28783701 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-amd64.bin
    62437893   27342472 2022-10-11 Planet/IPFS/go-ipfs-executables/ipfs-arm64.bin
    11005024   11003931 <present>  Planet/Labs/Icon Gallery/Icons/tier2.zip
     9225100    9225233 <present>  Planet/Labs/Icon Gallery/Icons/tier3.zip
    11586800    6243872 2022-07-20 Planet/IPFS/go-ipfs-executables/fs-repo-migrations-amd64
    11190496    5992917 2022-07-20 Planet/IPFS/go-ipfs-executables/fs-repo-migrations-arm64
     3648387    3599562 <present>  Screenshots/planetable.eth.png
     3028464    1863744 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-14-to-15_amd64
     3000352    1765206 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-14-to-15_arm64
     2720336    1666619 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-12-to-13_amd64
     2698528    1653893 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-13-to-14_amd64
     2670480    1563164 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-12-to-13_arm64
     2652384    1552732 <present>  Planet/IPFS/fs-repo-migrations/fs-repo-13-to-14_arm64
     1438642    1352107 <present>  Screenshots/vitalik.eth.png
     1208368    1163191 <present>  Technotes/Images/xcode15_build_issue.png
    31522127    1135696 <present>  Planet.xcodeproj/project.pbxproj
    ...

du -h: size of a fresh clone:

...
117M    ./.git/lfs
518M    ./.git
657M    .

Run this command to strip the largest IPFS files that are not in LFS:

git filter-repo \
--path Planet/IPFS/go-ipfs-executables/ipfs-amd64 \
--path Planet/IPFS/go-ipfs-executables/ipfs-arm64 \
--path Planet/Assets.xcassets/IPFS-GO.dataset/ipfs \
--path Planet/Assets.xcassets/IPFS-GO-ARM.dataset/ipfs \
--path Planet/IPFS/ipfs-executables/ipfs-amd64-0.28.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-amd64-0.28.bin \
--path Planet/IPFS/ipfs-executables/ipfs-arm64-0.28.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-arm64-0.28.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-amd64.bin \
--path Planet/IPFS/go-ipfs-executables/ipfs-arm64.bin \
--invert-paths

du -h: after running that command:

...
117M    ./.git/lfs
182M    ./.git
321M    .
livid commented 5 months ago

This issue does not block #365

livid commented 5 months ago

After #365 is merged, tested, and released as an Insider build, I will return to this issue to clean this tech debt.