search

skycoin / cx

A new programming language for blockchain.
Other
43 stars 48 forks source link

Clean git history from binaries #416

Closed jdknives closed 3 years ago

jdknives commented 3 years ago

The repo is currently more than 200MB large. Most of the data is stored in git directories of the repo which makes me suspect we pushed large or multiple binaries to history which are not tracked. We need to clean up the history to get the repo to a reasonable size.

This requires admin privileges as it will require rewriting git history.

SkycoinSynth commented 3 years ago 
All sizes are in kB's. The pack column is the size of the object, compressed, inside the pack file.
size   pack  SHA                                      location
12543  5522  5724995dbd7d1dfd6629eef8ec284f2ed41deb23  ide/cmd/leaps/leaps
12457  5518  ab23e9410da8bc298b59c0fe82d711c2d9fe7b6d  ide/cmd/leaps/leaps
8192   3484  40c5b253ebed6baf809ea1ae74ccd0982b02b35b  vendor/github.com/skycoin/skycoin/src/visor/testdata/data.db.nosig
6463   3560  0ec4b8c447f5773c886df1c439c75c762fd40009  benchmarks/test-lexer/test-toks.txt
4435   1930  28c8ada6f8a2ec76bd1c803f63ed00a2cc3f45fb  src/golike/cx.nn
3502   215   baa29e5deba72a6b475159d27df0e2052e04722c  object-explorer/static/vendors/moment/min/tests.js
2649   442   8703f50d3b7f4f898a395feed7060efdf68e43c8  dist/vendor.bundle.js
2584   865   4b6c0f76a0dbf009ba1b3b7380204e1ae0c34ab2  vendor/github.com/skycoin/skycoin/src/cipher/secp256k1-go/secp256k1-go2/z_consts.go
2237   260   1fd6c1003ecf6f8b697ee1c563ac1b0d07b4b8a7  vendor/github.com/go-gl/gl/v3.2-compatibility/gl/package.go
2205   418   74b09d21bf5e954badf4d186573f219c3f810e5d  object-explorer/static/vendors/pdfmake/build/pdfmake.min.js.map
SkycoinSynth commented 3 years ago

Top 50 largest objects in git histoy

1fd6c1003ecf6f8b697ee1c563ac1b0d07b4b8a7 vendor/github.com/go-gl/gl/v3.2-compatibility/gl/package.go
6c873535be579fdb4ae11573fdf138d355f325e7 vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/deps/nuklear.h
4d61315d7bda4d98707bd4f9df0e2b7764703d3e vendor/github.com/mjibson/go-dsp/wav/float.wav
4b6c0f76a0dbf009ba1b3b7380204e1ae0c34ab2 vendor/github.com/skycoin/skycoin/src/cipher/secp256k1-go/secp256k1-go2/z_consts.go
0ec4b8c447f5773c886df1c439c75c762fd40009 benchmarks/test-lexer/test-toks.txt
8a90741e5c759bde3872fd0f333fbcfad1d25140 documentation/r7rs.pdf
212b8558810e37b533850380b201ca11221744e8 readme-images/cx-roadmap.jpg
10f5202c693fde615b4ed7bebcdb85400660e912 vendor/golang.org/x/text/unicode/norm/tables12.0.0.go
344ab7558dcbd246380feae15c75fd2ecd1c1828 vendor/github.com/skycoin/noise/vectors.txt
9ea1b421407d55d5a207b022a49402bc1924ce18 vendor/golang.org/x/text/unicode/norm/tables13.0.0.go
4f9db587dda7a8a6e367a1029e5bbefdb79ae5bb book/img/functions-of.png
420be114a464a224850519f46f9e1b85747977be book/img/functions-on.png
2b4cea5b94a1c00a808b784bfd850764c24b6d54 vendor/golang.org/x/sys/windows/zerrors_windows.go
d73c27344ce322d412519d63590eea4657214d04 vendor/github.com/skycoin/skycoin/src/gui/static/dist/assets/img/header.png
03a84019497c00f1a37b14d9311f6c2a8fe1e1fb vendor/github.com/skycoin/skycoin/src/gui/static/dist/assets/img/otc-background.jpg
855c845e538b65548118279537a04eab2ec6ef0d vendor/github.com/skycoin/skycoin/src/gui/static/dist/fontawesome-webfont.912ec66d7572ff821749.svg
aaac5536c61cdb83c7c3c20d9bef1e8051254f03 vendor/github.com/skycoin/skycoin/src/gui/static/dist/main.8954a15cb8309677c946.bundle.js
dcd9ede389d6d6f5c2ec4addb4749b1abe21ffaf vendor/github.com/skycoin/skycoin/src/visor/blockdb/testdata/blockchain-180.no-unspent-addr-index.db
40c5b253ebed6baf809ea1ae74ccd0982b02b35b vendor/github.com/skycoin/skycoin/src/visor/testdata/data.db.nosig
86fa6461318c7b77698f7b94ab213ecd14384179 vendor/github.com/skycoin/skycoin/src/visor/testdata/data.db.notxn
21ebfa75c52a0ee947ec9b23a0b7f49854eeb310 vendor/github.com/skycoin/skycoin/src/visor/testdata/data.db.nouxout
47780f858d664cf4086da60a86603abae2d64481 vendor/github.com/skycoin/skycoin/src/visor/testdata/data.db.ok
01c28162fec9cac8247abda51f95e7d2cdc6af12 vendor/github.com/go-gl/gl/v2.1/gl/package.go
5724995dbd7d1dfd6629eef8ec284f2ed41deb23 ide/cmd/leaps/leaps
9f9cc4e5b73f4b2f752f47190cceda2681cd0641 vendor/github.com/go-gl/gl/v2.1/gl/package.go
ab23e9410da8bc298b59c0fe82d711c2d9fe7b6d ide/cmd/leaps/leaps
ec077f95f4c19f2548ad0d65604dea98aa7521c8 object-explorer/static/vendors/Chart.js/dist/Chart.bundle.js
f010c82d113c25336f5ae7df32ce5de0caaddbd2 object-explorer/static/vendors/bootstrap/dist/css/bootstrap.css.map
4f1d7bb4d784131853bd97816205889beb9d1f15 object-explorer/static/vendors/datatables.net/js/jquery.dataTables.js
a72e2f46712c323d5af04b3ed608984237f9c3f4 object-explorer/static/vendors/echarts/dist/echarts.common.js
e89d4a23b097cee26ee2d01ab06217dab1ec6511 object-explorer/static/vendors/echarts/dist/echarts.js
61e0b331256666e21be700420e151e25f00a6b5e object-explorer/static/vendors/echarts/dist/echarts.min.js
39a8c748602a0b5c4293815436303f2468b35aad object-explorer/static/vendors/echarts/dist/echarts.simple.js
098dcc6b4de3868f4fa22c4fd2d1b2826e152e8c object-explorer/static/vendors/echarts/map/js/world.js
377f3f599cf1cd13db919b08214332eb8a5558bf object-explorer/static/vendors/echarts/map/json/world.json
8b66187fe067c3aa389ce8c98108f349ceae159c object-explorer/static/vendors/font-awesome/fonts/fontawesome-webfont.svg
cf696b2996013bee497cbdb8ddc0dbe22079276c object-explorer/static/vendors/fullcalendar/dist/fullcalendar.js
355900810e40fb68d47c1b3773d6b15b829a6b6d object-explorer/static/vendors/jquery.inputmask/extra/phone-codes/phone-ru.js
baa29e5deba72a6b475159d27df0e2052e04722c object-explorer/static/vendors/moment/min/tests.js
5f2ffb1591e9850a13c9aa23508d9d633fcd3266 object-explorer/static/vendors/pdfmake/build/pdfmake.js
41b168d9d529fc616209e2f5ef3c6bb47698ceec object-explorer/static/vendors/pdfmake/build/pdfmake.min.js
74b09d21bf5e954badf4d186573f219c3f810e5d object-explorer/static/vendors/pdfmake/build/pdfmake.min.js.map
f5fd30a82e20973d145d590eb7a8327d436a4d31 object-explorer/static/vendors/pdfmake/build/vfs_fonts.js
d6f07aee15488126b7056c17245492f3dd7dd542 cx-roadmap.png
8703f50d3b7f4f898a395feed7060efdf68e43c8 dist/vendor.bundle.js
b5c8abaff39c0fcfae22daaefdbd97f5c76a856a dist/vendor.bundle.js.map
f3bf9f04ed0ef17bb052dd3c125ac3e4b04a8695 cxweb/package-lock.json
995313b4c80d9e3a183d1ac64b8d9009d477e7eb src/golike/y
28c8ada6f8a2ec76bd1c803f63ed00a2cc3f45fb src/golike/cx.nn
fe59c5fda57294634e0b39275ef144ce444d7290 documentation/specification/r7rs.pdf
SkycoinSynth commented 3 years ago 
du -sh * | sort -rh
 31M    goyacc
 21M    vendor
 14M    bin
1.2M    examples
1000K   tests
552K    cxgo
536K    cx
244K    docs
188K    cxfx
100K    benchmarks
 72K    LanguageGuide.md
 48K    cmd
 32K    gocraft.db
 24K    CompilerDevelopment.md
 16K    CONTRIBUTING.md
 16K    CHANGELOG.md
 12K    lib
 12K    go.sum
 12K    development
 12K    cx-setup.bat
8.0K    README.md
4.0K    go.mod
4.0K    Makefile
4.0K    LICENSE
SkycoinSynth commented 3 years ago 
du -h . | sort -rh

144M    .
 74M    ./.git
 73M    ./.git/objects
 66M    ./.git/objects/pack
 31M    ./goyacc
 29M    ./goyacc/y/testdata
 29M    ./goyacc/y
 28M    ./goyacc/y/testdata/ok
 21M    ./vendor
 14M    ./bin
 11M    ./vendor/github.com
 10M    ./vendor/golang.org/x
 10M    ./vendor/golang.org
9.3M    ./vendor/golang.org/x/sys
8.2M    ./vendor/golang.org/x/sys/unix
5.6M    ./vendor/github.com/go-gl
4.6M    ./.git/objects/46
3.4M    ./vendor/github.com/go-gl/glfw/v3.3/glfw
3.4M    ./vendor/github.com/go-gl/glfw/v3.3
3.4M    ./vendor/github.com/go-gl/glfw
3.2M    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw
2.8M    ./vendor/github.com/skycoin/skycoin/src
2.8M    ./vendor/github.com/skycoin/skycoin
2.8M    ./vendor/github.com/skycoin
2.7M    ./vendor/github.com/skycoin/skycoin/src/cipher
2.6M    ./vendor/github.com/skycoin/skycoin/src/cipher/secp256k1-go/secp256k1-go2
2.6M    ./vendor/github.com/skycoin/skycoin/src/cipher/secp256k1-go
2.2M    ./vendor/github.com/go-gl/gl/v3.2-compatibility/gl
2.2M    ./vendor/github.com/go-gl/gl/v3.2-compatibility
2.2M    ./vendor/github.com/go-gl/gl
1.9M    ./vendor/github.com/mjibson/go-dsp/wav
1.9M    ./vendor/github.com/mjibson/go-dsp
1.9M    ./vendor/github.com/mjibson
1.7M    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/deps
1.3M    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/src
1.2M    ./examples
1.1M    ./vendor/golang.org/x/sys/windows
1000K   ./tests
760K    ./goyacc/y/testdata/dev
720K    ./goyacc/mathutil
552K    ./vendor/golang.org/x/mobile
552K    ./cxgo
548K    ./examples/opengl
536K    ./cx
420K    ./goyacc/parser
400K    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/deps/glad
324K    ./goyacc/parser/yacc
316K    ./vendor/golang.org/x/exp
288K    ./vendor/golang.org/x/mobile/gl
288K    ./vendor/golang.org/x/exp/shiny
288K    ./examples/opengl/textures
276K    ./vendor/golang.org/x/exp/shiny/driver
264K    ./goyacc/fileutil
244K    ./docs
236K    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/include/GLFW
236K    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/include
228K    ./docs/images
204K    ./cxgo/parser
192K    ./goyacc/parser/yacc/testdata
188K    ./cxfx
184K    ./vendor/github.com/golang/freetype
184K    ./vendor/github.com/golang
176K    ./vendor/golang.org/x/exp/shiny/driver/internal
172K    ./vendor/github.com/sirupsen/logrus
172K    ./vendor/github.com/sirupsen
168K    ./examples/opengl/fonts
168K    ./examples/book
156K    ./goyacc/golex
152K    ./cxgo/cxgo0
144K    ./goyacc/scanner
140K    ./cxgo/actions
128K    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/deps/mingw
128K    ./examples/buggy-examples
124K    ./examples/affordances
120K    ./vendor/golang.org/x/mobile/app
120K    ./vendor/github.com/mattn
116K    ./goyacc/biggfft
108K    ./vendor/golang.org/x/exp/shiny/driver/internal/x11key
108K    ./vendor/github.com/golang/freetype/truetype
108K    ./.git/objects/22
104K    ./goyacc/scanner/yacc
104K    ./goyacc/fileutil/falloc
100K    ./vendor/golang.org/x/exp/shiny/driver/gldriver
100K    ./goyacc/lexer
100K    ./benchmarks
 88K    ./goyacc/lex
 88K    ./.git/objects/21
 84K    ./goyacc/parser/nquads
 80K    ./goyacc/golex/examples
 72K    ./.git/objects/ab
 72K    ./.git/objects/69
 68K    ./vendor/golang.org/x/crypto
 68K    ./cx/base
 64K    ./vendor/github.com/mattn/go-colorable
 60K    ./vendor/github.com/skycoin/gltext
 60K    ./goyacc/fileutil/storage
 60K    ./examples/buggy-examples/opengl
 60K    ./examples/affordances/affordances-on
 60K    ./examples/affordances/affordances-of
 56K    ./vendor/github.com/mattn/go-isatty
 56K    ./vendor/github.com/magefile/mage
 56K    ./vendor/github.com/magefile
 56K    ./.git/objects/75
 52K    ./vendor/golang.org/x/mobile/exp/audio/al
 52K    ./vendor/golang.org/x/mobile/exp/audio
 52K    ./vendor/golang.org/x/mobile/exp
 52K    ./vendor/golang.org/x/exp/shiny/driver/internal/win32
 52K    ./vendor/golang.org/x/crypto/ssh/terminal
 52K    ./vendor/golang.org/x/crypto/ssh
 52K    ./.git/refs
 52K    ./.git/objects/97
 52K    ./.git/logs
 48K    ./vendor/github.com/golang/freetype/raster
 48K    ./goyacc/sortutil
 48K    ./cmd
 48K    ./benchmarks/cx-vs-golang
 48K    ./.git/objects/d6
 48K    ./.git/hooks
 44K    ./vendor/golang.org/x/image
 44K    ./cmd/cx
 44K    ./.git/logs/refs
 40K    ./vendor/github.com/satori/go.uuid
 40K    ./vendor/github.com/satori
 40K    ./goyacc/golex/lex
 40K    ./goyacc/golex/examples/calc
 40K    ./benchmarks/cx-vs-python
 40K    ./.git/objects/72
 40K    ./.git/objects/70
 40K    ./.git/objects/18
 36K    ./vendor/golang.org/x/mobile/event
 36K    ./goyacc/scanner/nquads
 36K    ./goyacc/mathutil/mersenne
 36K    ./examples/blockchain
 36K    ./.git/refs/remotes/origin
 36K    ./.git/refs/remotes
 36K    ./.git/objects/fb
 36K    ./.git/objects/62
 36K    ./.git/logs/refs/remotes/origin
 36K    ./.git/logs/refs/remotes
 32K    ./vendor/github.com/magefile/mage/mg
 32K    ./examples/buggy-examples/affordances
 32K    ./.git/objects/6b
 32K    ./.git/objects/61
 32K    ./.git/objects/3b
 32K    ./.git/objects/39
 32K    ./.git/objects/05
 28K    ./vendor/github.com/skycoin/skycoin/src/cipher/encoder
 28K    ./vendor/github.com/mgutz/ansi
 28K    ./vendor/github.com/mgutz
 28K    ./vendor/github.com/jinzhu/copier
 28K    ./vendor/github.com/jinzhu
 28K    ./examples/data-structures
 28K    ./.git/objects/ea
 28K    ./.git/objects/b7
 28K    ./.git/objects/af
 28K    ./.git/objects/a6
 28K    ./.git/objects/90
 24K    ./vendor/github.com/skycoin/skycoin/src/util/logging
 24K    ./vendor/github.com/skycoin/skycoin/src/util
 24K    ./vendor/github.com/skycoin/skycoin/src/cipher/base58
 24K    ./goyacc/fileutil/hdb
 24K    ./goyacc/biggfft/benchmarks
 24K    ./.git/objects/e7
 24K    ./.git/objects/dd
 24K    ./.git/objects/ce
 24K    ./.git/objects/c4
 24K    ./.git/objects/a7
 24K    ./.git/objects/87
 24K    ./.git/objects/73
 24K    ./.git/objects/64
 24K    ./.git/objects/30
 24K    ./.git/objects/2f
 24K    ./.git/objects/29
 24K    ./.git/objects/1d
 24K    ./.git/objects/17
 24K    ./.git/objects/06
 20K    ./vendor/golang.org/x/mobile/asset
 20K    ./vendor/golang.org/x/mobile/app/internal/callfn
 20K    ./vendor/golang.org/x/mobile/app/internal
 20K    ./cxgo/cxlexer
 20K    ./.git/objects/d3
 20K    ./.git/objects/bd
 20K    ./.git/objects/aa
 20K    ./.git/objects/a0
 20K    ./.git/objects/84
 20K    ./.git/objects/4f
 20K    ./.git/objects/20
 20K    ./.git/objects/1e
 20K    ./.git/objects/15
 20K    ./.git/objects/07
 16K    ./vendor/golang.org/x/mobile/internal/mobileinit
 16K    ./vendor/golang.org/x/mobile/internal
 16K    ./vendor/golang.org/x/image/math
 16K    ./tests/issue-177
 16K    ./goyacc/golex/examples/go
 16K    ./examples/http-server
 16K    ./examples/control-flow
 16K    ./cxgo/cxgo
 16K    ./.git/objects/f3
 16K    ./.git/objects/f2
 16K    ./.git/objects/f1
 16K    ./.git/objects/ee
 16K    ./.git/objects/d8
 16K    ./.git/objects/ca
 16K    ./.git/objects/c5
 16K    ./.git/objects/be
 16K    ./.git/objects/96
 16K    ./.git/objects/88
 16K    ./.git/objects/85
 16K    ./.git/objects/80
 16K    ./.git/objects/7d
 16K    ./.git/objects/7a
 16K    ./.git/objects/57
 16K    ./.git/objects/56
 16K    ./.git/objects/50
 16K    ./.git/objects/4d
 16K    ./.git/objects/4a
 16K    ./.git/objects/49
 16K    ./.git/objects/47
 16K    ./.git/objects/31
 16K    ./.git/objects/28
 16K    ./.git/objects/25
 16K    ./.git/objects/16
 16K    ./.git/objects/13
 16K    ./.git/objects/10
 16K    ./.git/objects/0e
 16K    ./.git/objects/0d
 16K    ./.git/objects/0b
 12K    ./vendor/golang.org/x/mobile/event/key
 12K    ./vendor/golang.org/x/image/math/fixed
 12K    ./vendor/golang.org/x/image/font
 12K    ./vendor/golang.org/x/exp/shiny/screen
 12K    ./vendor/golang.org/x/exp/ebnf
 12K    ./vendor/github.com/skycoin/skycoin/src/cipher/ripemd160
 12K    ./vendor/github.com/magefile/mage/sh
 12K    ./vendor/github.com/go-gl/glfw/v3.3/glfw/glfw/deps/vs2008
 12K    ./tests/test-workspace/src/math
 12K    ./tests/test-workspace/src
 12K    ./tests/test-workspace
 12K    ./lib
 12K    ./goyacc/golex/examples/numbers
 12K    ./goyacc/golex/examples/c-like
 12K    ./examples/project-example
 12K    ./development
 12K    ./cxgo/api
 12K    ./benchmarks/cx-vs-golang/factorial-recursive
 12K    ./benchmarks/cx-vs-golang/factorial-iterative
 12K    ./benchmarks/cx-vs-golang/digital-root
 12K    ./benchmarks/cx-vs-golang/ackermann
SkycoinSynth commented 3 years ago

I think it is all vendor folder.