search

JohanSchott / impurityModel

Calculate many-body states of an impurity Anderson model and spectra (e.g. XPS, XAS, RIXS, NIXS)
MIT License
22 stars 2 forks source link

Reduce .git diskspace by using git lfs #59

Open JohanSchott opened 3 weeks ago

JohanSchott commented 3 weeks ago

The folder that keeps all history, .git, is currently about 278 MB. The current master is about 60 MB, and the biggest file, impurityModel/test/referenceOutput/Ni_NiO_50bath/spectra.h5, is about 57 MB. This file is currently not tracked with git lfs.

It would be nice to reduce the diskspace usage, perhaps by using git lfs migrate import. But to reduce the size of .git I think one needs to modify also the git-history (to make it look like it has only been a git lfs file). I image this potentially can reduce the size of .git from 278 MB to something close to 60 MB.

But is it worth rewriting the git-history in order to reduce the size of .git? Or is there another way to reduce the size of .git?

JohanSchott commented 3 weeks ago

Reference: https://github.com/git-lfs/git-lfs/wiki/Tutorial

kalvdans commented 3 weeks ago

I think a history that is 4x the size of current master is not that strange and no need to drop the history.

If we can reduce the unit test size to, say, checking 30-40 key values, it would be another story and we can think about rewriting history. We recently did that at our job (internal ref) by following the guide for gitlab.

JohanSchott commented 2 weeks ago

Nice input!

yeah, perhaps reducing the size with 4x size is not enough to justify a rewrite of the history.

Right now all elements in all 18 datasets in impurityModel/test/referenceOutput/Ni_NiO_50bath/spectra.h5 are tested in the unit-test. I like the suggestion of reducing all those checks to 30-40 spot checks but I'm not sure how to best do that without loosing too much confidence that nothing has changed.