Mic92 / nixpkgs-review

Review pull-requests on https://github.com/NixOS/nixpkgs
MIT License
356 stars 59 forks source link

respect shallow/grafted nixpkgs repositories #267

Open xfnw opened 2 years ago

xfnw commented 2 years ago

if you attempt to nixpkgs-review pr on a shallow copy of nixpkgs, it will try to download the entire ~4gb of git history

Mic92 commented 2 years ago

I don't see a way around this. It currently uses git fetch to download the history of both master and the PR and than needs to perform a merge of one into the other

SuperSandro2000 commented 2 years ago

it will try to download the entire ~4gb of git history

For me the entire history is just 1.3 GB big.

We could probably modify the fetch commands to only fetch what we really need but it is a pain to work with shallow repositories since certain things done with the history will just fail in some situations and executing multiple fetches for every user is probably slowing down things more.

xfnw commented 2 years ago

For me the entire history is just 1.3 GB big.

huh... do you have some sort of compression enabled? mine seems to be a lot larger

[xfnw@feesh:~]$ du -sh nixpkgs
3.7G    nixpkgs

[xfnw@feesh:~]$ du -sh nixpkgs/.git
3.6G    nixpkgs/.git
SuperSandro2000 commented 2 years ago

Try

du -sh .git
git gc --aggressive
git gc
git repack -Ad
git prune
du -sh .git
lolbinarycat commented 5 months ago

@SuperSandro2000

We could probably modify the fetch commands to only fetch what we really need but it is a pain to work with shallow repositories since certain things done with the history will just fail in some situations and executing multiple fetches for every user is probably slowing down things more.

what about an arbitrarily large but still much smaller than the whole repository amount?

for example, fetching only the last 10000 commits?

waiting 10 minutes is still a lot better than waiting 2 hours, and people using a shallow clone are likely to have a a slow network connection anyways.