I don't have any logs right now, but this one should be easy enough to replicate.
If you crawl a subreddit, and domain the program is trying to download an image from has expired/doesn't exist, the program will crash.
The same thing seems to happen if the domain exist, but is offline (doesn't respond).
Would it be possible to make the program just skip these entries after a few retries, instead of completely shutting down?
Hi,
I don't have any logs right now, but this one should be easy enough to replicate. If you crawl a subreddit, and domain the program is trying to download an image from has expired/doesn't exist, the program will crash.
The same thing seems to happen if the domain exist, but is offline (doesn't respond). Would it be possible to make the program just skip these entries after a few retries, instead of completely shutting down?