By default, only makes a 365-day shallow clone of gnulib

[rrthomas]

A project of mine is breaking because it uses an older commit. As gnulib is a source-only library, there should be no reason to use a newer version unless new fixes/features are required. Hence, if anything, encourage users to have a reusable local clone.

If this is acceptable, I'll happily file a PR to remove the --shallow argument.

[praiskup]

That --depth (365 commits) argument is only optimization -- because it doesn't seem to be wise to download full gnulib repository (several tens of MB; I have now connection like 750kB/s and it takes a really long time).

Could we use configurable --shallow-since=<date> (e.g. gnulib_clone_since?), and fallback to 365 commits as it was before? Small inconvenience is that this argument isn't in e.g. git 1.8 so - if such configuration exists - I'd fallback to full deep clone if user is using old git.

[rrthomas]

Is that situation (slow/unreliable internet connection) not already addressed by having a local clone of gnulib and setting GNULIB_SRCDIR?

[praiskup]

Not really for random contributor doing the ./bootstrap for the first time (they might have no idea what gnulib is).

[rrthomas]

Right, but that's the same problem as for checking out the main repo of whatever project it is. Why treat gnulib specially? It would be easy for example to print out a warning if GNULIB_SRCDIR is not set.

[praiskup]

Hmpfs, when user has ./bootstrap script, he needs to have the main project cloned already (and it is up to user to terminate the intitial clone if the connection is slow, and fallback to shallow clone). But ./bootstrap is expected to clone gnulib automatically, and it makes a pretty big difference for me (consider that you need only gnulib newer than 2019-02-19):

12:14:55 /tmp$ git clone --shallow-since=2019-02-19 git://git.savannah.gnu.org/gnulib.git gnulib-shallow
Cloning into 'gnulib-shallow'...
remote: Counting objects: 10388, done.
remote: Compressing objects: 100% (8996/8996), done.
remote: Total 10388 (delta 5669), reused 2772 (delta 1369)
Receiving objects: 100% (10388/10388), 8.35 MiB | 687.00 KiB/s, done.
Resolving deltas: 100% (5669/5669), done.

vs

12:15:25 /tmp$ git clone git://git.savannah.gnu.org/gnulib.git gnulib-full
Cloning into 'gnulib-full'...
remote: Counting objects: 202121, done.
remote: Compressing objects: 100% (27012/27012), done.
remote: Total 202121 (delta 175295), reused 201791 (delta 175043)
Receiving objects: 100% (202121/202121), 41.07 MiB | 755.00 KiB/s, done.
Resolving deltas: 100% (175295/175295), done.
12:17:31 /tmp$

[praiskup]

It would be easy for example to print out a warning if GNULIB_SRCDIR is not set.

I don't want to complicate the random contributor life in random dependant project (by default), but if whatever working for you was added as opt-in (configurable by bootstrap.conf) -- it would be acceptable I think; with opt-in, it is then your choice (the main project maintainer) choice that you enforce the full gnulib clone.

[rrthomas]

OK, then to avoid the problem I had (which was quite hard to track down), bootstrap should check out the correct commit, not do some arbitrary-length shallow clone, which will always go out of date. I just had a look to find out how to do a shallow clone to a particular commit but I diidn't get very far yet. Is that --shallow-exclude? (I mean, that bootstrap should look up the correct commit of gnulib, and use that.)

[rrthomas]

(By the way, the current situation is complicating the life of a random contributor: I found this problem precisely because a contributor to one of my projects was complaining that he could not check it out. It took me a long time to track down the source of the problem!)

[praiskup]

OK, then to avoid the problem I had (which was quite hard to track down),

I agree that it is a problem, and I don't underestimate it. Happened to me as well.

bootstrap should check out the correct commit, not do some arbitrary-length shallow clone, which will always go out of date.

Agreed, I think that if you specified the --shallow-since=<DATE> explicitly, you have this totally under control. The later the script is run then, the deeper the clone will be (but that is what you expected).

I think that we can simply do the --shallow-since, and if the option isn't available (or maintainer did not specify <DATE>) -- do the full clone, to not overcomplicate the logic...

I just had a look to find out how to do a shallow clone to a particular commit but I diidn't get very far yet. Is that --shallow-exclude? (I mean, that bootstrap should look up the correct commit of gnulib, and use that.)

To actually find the commit, I think you have to do the clone. Dunno.

[praiskup]

By the way, the current situation is complicating the life of a random contributor

Yes, I completely agree with you.... and I'd be glad to have this resolved. Will you take a look at the patch personally?

[rrthomas]

It seems that Savannah does not support --shallow-exclude, and also it requires the parent commit, and I don't know how you would calculate that (so I guess it would still have to be configured); therefore, I guess --shallow-since is good enough. I'll have a look.

[rrthomas]

See PR #21.