nilfoer / gwaripper

Tool for conveniently downloading audios from r/gonewildaudio and similar subreddits
MIT License
30 stars 5 forks source link

Incomplete downloads from erocast.me #7

Closed unrealtournament closed 2 years ago

unrealtournament commented 2 years ago

The following audio (NSFW) https://erocast.me/track/3958/shy-vampire-roomate, taken from this reddit post https://old.reddit.com/r/gonewildaudio/comments/wu0csy/f4m_your_shy_vampire_roommate_wants_to_suck_your/ downloads an mp4 file that's 4 minutes long when the real audio length is 14:01 minutes.

Command issued:

python3 gwaripper-runner.py links https://erocast.me/track/3958/shy-vampire-roomate

Download log:

2022-08-23 04:00:54,139 - gwaripper - INFO   - Processing URL 1 of 1: https://erocast.me/track/3958/shy-vampire-roomate
2022-08-23 04:00:55,066 - gwaripper.extractors.base - DEBUG  - Getting html done!
2022-08-23 04:00:55,067 - gwaripper - INFO   - Downloading: shy vampire roomate.mp4..., File 1 of 1
2022-08-23 04:00:55,067 - gwaripper - INFO   - Wating for ffmpeg to finish...
2022-08-23 04:01:08,588 - gwaripper - INFO   - Download report was written to folder _reports

HTML report says DOWNLOADED and NO_ERRORS.

I've tried downloading the m3u8 file served in the website manually with youtube-dl, it seems like the website is throwing a 429 error (Too Many Requests) whenever you try to download the m3u fragments too quickly, the first 3 ones download correctly and then these errors start popping up:

ERROR: [generic] Unable to download webpage: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: 'Too Many Requests'>);

Additional note: I've tried it with a US VPN and now it downloaded 13 minutes before starting to throw 429 errors, however a few minutes in the middle were silent. So it seems inconsistent and dependent on the source IP, probably better to catch the errors and retry the failed downloads in some way.

nilfoer commented 2 years ago

Thank you for reporting this! I naively assumed that ffmpeg would have a non-zero exit code when this happens.

Now we're downloading the playlist and parts ourselves with a proper backoff and retries etc. (see 8cd213b)

unrealtournament commented 2 years ago

Thank you, works wonderfully now! Closing this since its fixed.