tsudoko / pullcord

Discord archiver
The Unlicense
60 stars 11 forks source link

Too many non-fatal errors are treated as fatal #21

Open tsudoko opened 4 years ago

tsudoko commented 4 years ago

Examples: #3, #7, #9, #10, #18, #20 There is a good reason for the current default behavior: skipping many of these errors would leave logs permanently incomplete. If you re-run pullcord, it continues downloading data from where it left off without checking if the already downloaded logs don't have gaps. This is in order to avoid re-requesting entire message history on each run, which would make incremental downloads extremely slow, and arguably, not very incremental.

The default behavior is perfectionist and tries to clone guild state as closely as possible, aborting if it can't save absolutely everything that can be saved1. As the above issues illustrate, this often leads to early failures and inability to download the rest of the guild at all.

One way of solving this could be marking some errors as non-fatal and adding a command line option that would skip those instead of aborting. It would be still useful to have them reported back here, though, as many of these errors indicate some kind of bug in pullcord. Not crashing could make people less likely to report bugs.

  1. Well, to be exact, this is no longer the case as of d6f3ad62eb526e9d40d1f203ee0ea3009c3dac12 because that error can happen very often, but ideally we'd abort in that case as well.