search

AlexCSDev / PatreonDownloader

Powerful tool for downloading content posted by creators on patreon.com. Supports content hosted on patreon itself as well as external sites (additional plugins might be required).
MIT License
917 stars 95 forks source link

Downloader stops at 3900/ #212

Open ThunderousLeft opened 7 months ago

ThunderousLeft commented 7 months ago

No error produced - the downloader simply stopped downloading when it hit 3900/4600+ files, then stopped downloading when it hit 1252/1975 files, and consistently doesnt grab those last 700 downloads.

I was hoping to get your help to understand what is going on here

ThunderousLeft commented 7 months ago

image

DavidArchibald commented 4 months ago

The problem is this snippet: https://github.com/AlexCSDev/UniversalDownloaderPlatform/blob/8eec8693944f5db4eb36e265f102dfe0889600e0/UniversalDownloaderPlatform.Engine/DownloadManager.cs#L91-L104

When a DownloadException happens OnFileDownloaded is still called (with a flag saying it failed) but any other error doesn't call OnFileDownloaded which is used for the counting. It seems you have roughly 700 errors which explains the undercounting.

RisaDev commented 4 months ago

That shouldn't be the case because any error other than DownloadException is not being handled anywhere and supposed to crash the application from what I can see.

DavidArchibald commented 4 months ago

Yeah I forgot that handling other errors was something I added in a fork so that solved it for me in that case.

The undercounting is still going to apply here though: https://github.com/AlexCSDev/UniversalDownloaderPlatform/blob/8eec8693944f5db4eb36e265f102dfe0889600e0/UniversalDownloaderPlatform.Engine/DownloadManager.cs#L56-L66

It returns (ending the task) when it's not a valid url or it's a blacklisted URL without calling OnFileDownloaded either which means it never reaches the same event I was referring to last time: https://github.com/AlexCSDev/PatreonDownloader/blob/d201c1434a62df19502043dbc102ce07be54b8a8/PatreonDownloader.App/Program.cs#L210-L217