search

ArchiveTeam / ArchiveBot

ArchiveBot, an IRC bot for archiving websites
http://www.archiveteam.org/index.php?title=ArchiveBot
MIT License
357 stars 72 forks source link

Refused connections are not retried #464

Open JustAnotherArchivist opened 4 years ago

JustAnotherArchivist commented 4 years ago

ArchiveBot does not retry [Errno 111] Connection refused errors. I think it should and consider this a bug. This simply requires adding the wpull option --retry-connrefused.

JustAnotherArchivist commented 1 year ago

Real-world example of something that effectively can't be archived due to this without ridiculous manual effort: ShoutWiki. It refuses connections in a pretty much random manner, which then wrecks any recursive crawls.