I am not sure how frequent and in under which circumstances but scanning a mirrorbrain mirror can fail.
Obviously, we don't want to halt the scanning process (because currently it loops over each folder over each mirror sequentially – could be different though) but we should at least report somehow that something went wrong.
This is following wikimedia mirror changing its URL (HTTP 301) without informing us, leading to scans all failing for wikimedia, disabling this major mirror in mirrorbrain ; thus increasing traffic to other mirrors ; (at least one) of which disabled itself as unable to respond to this increased traffic.
I am not sure how frequent and in under which circumstances but scanning a mirrorbrain mirror can fail. Obviously, we don't want to halt the scanning process (because currently it loops over each folder over each mirror sequentially – could be different though) but we should at least report somehow that something went wrong.
This is following wikimedia mirror changing its URL (HTTP 301) without informing us, leading to scans all failing for wikimedia, disabling this major mirror in mirrorbrain ; thus increasing traffic to other mirrors ; (at least one) of which disabled itself as unable to respond to this increased traffic.