evalEmpire / gitpan

Git repositories for all of CPAN
http://github.com/gitpan
73 stars 12 forks source link

Add the ability to update periodically #109

Closed schwern closed 10 years ago

schwern commented 10 years ago

Right now, to update Gitpan you iterate through all 35,000 distributions again. This is, obviously, a bit wasteful. It would be better to update releases after a date.

The timestamp is the newest release in the BackPAN::Index at the start of the run. It's better to have a bit of an overlap than to miss some.

schwern commented 10 years ago

Because the BackPAN index creator was still broken, the latest full import will be since 2014-10-28T22:08:49 aka 1414534129. Keep that in mind when you run the update.