The per host crawler logs are available via http to be inspected by the
mirror admins if necessary. The current setup to daily rotate those log
files overwrites the log files with (empty) files so fast that most of
the times the log files are empty and have no real value for the mirror
admins. By rotating those log files only once per week the chances are
much higher that they might include useful information.
pypingou
commented
9 years ago
The per host crawler logs are available via http to be inspected by the mirror admins if necessary. The current setup to daily rotate those log files overwrites the log files with (empty) files so fast that most of the times the log files are empty and have no real value for the mirror admins. By rotating those log files only once per week the chances are much higher that they might include useful information.