szabgab / CPAN-Digger

CPAN Digger to analyze CPAN modules and other Perl-based applications
https://cpan-digger.perlmaven.com/
7 stars 3 forks source link

Reduce time spent updating git clones #31

Open szabgab opened 1 year ago

szabgab commented 1 year ago

We have cloned the git repositories of the 13,000 CPAN distributions that had a link to their git repo (out of the total of 40,000 distributions). We would like to keep the up to-date, but git pull takes about 0.250 sec even if there is nothing to update. This means about 1 hour of processing time for the 13K distributions. Most of them without any changes.

We don't want to use the new release as the trigger to update the clone because a developer might make several changes to a repo without releasing a new version to CPAN. (e.g. setting up GitHub Actions).