openpreserve / jpylyzer

JP2 (JPEG 2000 Part 1) validator and properties extractor. Jpylyzer was specifically created to check that a JP2 file really conforms to the format's specifications. Additionally jpylyzer is able to extract technical characteristics.
http://jpylyzer.openpreservation.org/
Other
69 stars 28 forks source link

Clean up / reduce size of git repo #105

Open bitsgalore opened 6 years ago

bitsgalore commented 6 years ago

Dev Effort

1D

Description

Currently > 200 MB, this is getting ridiculous.

Some clues:

https://confluence.atlassian.com/bitbucket/reduce-repository-size-321848262.html

carlwilson commented 4 years ago

I've done some experimentation here with some success. I started here: https://stackoverflow.com/questions/2116778/reduce-git-repository-size which took me here: https://stackoverflow.com/questions/685319/git-pull-error-unable-to-create-temporary-sha1-filename/685422#685422. I've managed to reduce the repository size to 92 MB by:

$ git-prune
$ git-gc --aggressive
$ git-repack
$ git-repack -a

I'm not yet convinced enough that there's no repercussions to force a push over the GitHub repo. This might best work as a maintenance task straight after the release when we can prune the branches back a little also.

carlwilson commented 4 years ago

@bitsgalore, I've been running scared of this all week but think I have a way forward. I'm going to fork as full a copy of the repo as I can before doing this and keep it somewhere safe. I'll then shrink the main repo, run a few tests here to make sure it works at a fairly superficial level. I'll then force the push over GitHub. My thinking is that if something comes up, we can use the archived fork to recover, though I don't think this will be necessary. Once you give me the go-ahead, I'll make a start.

carlwilson commented 4 years ago

@bitsgalore, hold that thought, I have a better way. I've forked to opf-attic and opf-labs and will shrink the opf-labs instance now so we can see what we think first.

Caution seems to be justified; I appear to have broken the opf-labs fork.