search

MalloyDelacroix / DownloaderForReddit

The Downloader for Reddit is a GUI application with some advanced features to extract and download submitted content from reddit.
GNU General Public License v3.0
505 stars 47 forks source link

Downloading stalls at 50%, can't restart download #134

Closed atorabli closed 3 years ago

atorabli commented 4 years ago

When I try to download all the posts for a user it will stall at 50% no matter how many posts there are. If I cancel the download and try again it won't download anything and will show as complete in the output window.

MalloyDelacroix commented 4 years ago

It sounds like you are probably actually downloading all of the content, and the problem is with the progress bar display. If you will attach your log file, I may be able to resolve this issue.

atorabli commented 4 years ago

I left it running overnight and the download was stuck at 26 of 279. The progress bar was near 50%. I tried a new user and faced the same thing, it got stuck on 9 of 85 for a few hours with the progress bar in the middle.

atorabli commented 4 years ago

The last user I tried to download just reached 58 of 260 and stalled.
DownloaderForReddit.log

MalloyDelacroix commented 4 years ago

I believe I have resolved this issue, but I have never been able to replicate it fully. If you would, please check to see if this issue persists with the new version.

atorabli commented 4 years ago

Seems like it's mainly stalling with large amounts of content and/or video downloads. I have to delete my entire user and sub database in order to test it. Anything that stalled before just shows as complete when I try to download again even though it might have stopped at 50/350. Is there a way to rectify that download issue without deleting the database? There was another user with this issue that was closed earlier today.

MalloyDelacroix commented 4 years ago

Under the "Download" menu, you can click "Run All Unfinished". This will extract any post in the database that has not been extracted (unless it was not extracted due to certain errors that cannot be resolved) and download any content that has not already been downloaded (again, certain errors not withstanding).

If that doesn't work: you can try to go under the user/subreddit settings or their containing list settings and set the date limit to a date that would pick up all of the posts that you want, and also uncheck "avoid duplicates". This should download anything that it doesn't already have in the database.

If neither of those options work, it likely means that it has downloaded everything that it can under the current settings. I would recommend double checking the settings to ensure that there are nothing in there preventing downloads that you want.

The only similar issue I see that was closed today was related to reddit videos. It shouldn't have been related to this issue. But if the items you are trying to download that are failing are all hosted on v.redd.it, then this issue is fixed with the latest release.

MalloyDelacroix commented 4 years ago

As a side note; if you want to test something on a clean database, you can rename your database file and run the application. It will create a new database file, and when you are done with it, you can delete or rename it, then change the name of the original back to "dfr.db". The app will always look for a database with that name.

atorabli commented 4 years ago

I deleted(renamed) the db a few times to see if the download was stopping at the same spot. On 5 tries the download would give an ERROR: Content does not exist, on the exact same post. One was from imgur and the other gfycat. I tried to disable downloads of gifs as that seems to be where it hangs but the settings won't hold. Tried saving and applying both and when I went back each time gifs were checked.

MalloyDelacroix commented 3 years ago

The issue that made almost all of the imgur links in your log file fail has been fixed, as well as the issue with the youtube video downloads. I haven't been able to figure out why these failures were halting the download, but maybe the new release will solve your problem. If it does not, please attach a new log file from the latest release. There should be more logging information available in it that will help.

atorabli commented 3 years ago

I can disable gifs now but I'm still running into issues with gfycat dead links. Would it be possible to force a user/subreddit to be removed from the db file completely when you remove it from the download list? Currently when I'm getting the error once removing the user then adding it back causes an issue where it will never download anything from that user/sub again.

I also noticed that when I this issue pops up and I stop the download then close the program it's still open in task manager. I don't see this happen when all the downloads finish properly even if I stop the download process by hitting the button.

MalloyDelacroix commented 3 years ago

I believe the underlying problem that was causing the downloads to stall has been resolved. I have also added the ability to completely delete users/subreddits, posts, content, and comments from the database.