ArchiveTeam / ArchiveBot

ArchiveBot, an IRC bot for archiving websites
http://www.archiveteam.org/index.php?title=ArchiveBot
MIT License
356 stars 72 forks source link

Pass TMPDIR to wpull if present to avoid jobs stalling on systems with small /tmp partition #491

Closed JustAnotherArchivist closed 3 years ago

JustAnotherArchivist commented 3 years ago

To expand on this, by default TMPDIR is not passed to wpull. Although wpull's temporary files all(?) end up in the cwd anyway, wpull still checks the temporary directory given by tempfile.gettempdir() on disk space monitoring. If /tmp is a separate and small partition, that then blocks wpull processes with no way to unblock.