search

jmbannon / ytdl-sub

Lightweight tool to automate downloading and metadata generation with yt-dlp
https://ytdl-sub.readthedocs.io
GNU General Public License v3.0
1.58k stars 60 forks source link

Show yt-dlp logs by default #992

Open TheSpyder opened 1 month ago

TheSpyder commented 1 month ago

I've busted something in my docker setup and some containers have 100% network failure. I'm not asking for help with that, I found a way to fix the network connection, but I think ytdl-sub should handle the situation better. The logs have no indication of such a critical failure and it took a day or two before I noticed I wasn't getting any new videos.

At the default log level everything appeared mostly normal. The only indication something might be wrong in the logs was that a few channels had expiring videos, and every run listed -1 or -2 for those channels without actually deleting anything. Then I realised the total was showing +9 every run without any downloads - and that + was equal to the number of expired videos.

Only after I increased the log level did it show the real issue:

[ytdl-sub] Validating subscriptions...
[ytdl-sub:subscription] Filtering subscriptions by name based on --match arguments
[ytdl-sub] Beginning subscription download for Skill Up
[ytdl-sub:yt-dlp] [youtube:tab] Extracting URL: https://www.youtube.com/@SkillUp/videos
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading webpage
[ytdl-sub:yt-dlp] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f937d25a0>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again). Retrying (1/3)...
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading webpage
[ytdl-sub:yt-dlp] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f937d13d0>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again). Retrying (2/3)...
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading webpage
[ytdl-sub:yt-dlp] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f937d3560>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again). Retrying (3/3)...
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading webpage
[ytdl-sub:yt-dlp] WARNING: [youtube:tab] Unable to download webpage: <urllib3.connection.HTTPSConnection object at 0x7f937c8e00>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again) (caused by TransportError("<urllib3.connection.HTTPSConnection object at 0x7f937c8e00>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again)")). Giving up after 3 retries
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading API parameters API JSON
[ytdl-sub:yt-dlp] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f937d1430>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again). Retrying (1/3)...
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading API parameters API JSON
[ytdl-sub:yt-dlp] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f937c9be0>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again). Retrying (2/3)...
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading API parameters API JSON
[ytdl-sub:yt-dlp] WARNING: [youtube:tab] <urllib3.connection.HTTPSConnection object at 0x7f937d3230>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again). Retrying (3/3)...
[ytdl-sub:yt-dlp] [youtube:tab] @SkillUp/videos: Downloading API parameters API JSON
[ytdl-sub:yt-dlp] ERROR: [youtube:tab] @SkillUp: Unable to download API page: <urllib3.connection.HTTPSConnection object at 0x7f937ca540>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again) (caused by TransportError("<urllib3.connection.HTTPSConnection object at 0x7f937ca540>: Failed to resolve 'www.youtube.com' ([Errno -3] Try again)"))
[ytdl-sub:downloader] Beginning downloads for https://www.youtube.com/@SkillUp/videos
[ytdl-sub] 
No files changed for Skill Up
[ytdl-sub] Download Summary:
Skill Up     0 0 0     30 ✔
Total: 1     0 0 0     30 Success

I suggest having a more critical error when there are network failures :)

I'm running ghcr.io/jmbannon/ytdl-sub:latest and the current image (it actually started after a docker pull, so initially I wondered if it was a ytdl-sub bug).

Replicating is easy - with a running container do docker network disconnect <container-network> <container-name>.

TheSpyder commented 1 month ago

After looking at the logs some more, I realise that it's apparently normal for the total expired videos to be a + in the totals row. So that part was normal, it's just that nothing was actually deleted due to the download error.

jmbannon commented 1 month ago

Maybe I'll make --log-level verbose the default, which shows yt-dlp logs and not by debug logs. Thanks for the suggestion @TheSpyder :slightly_smiling_face: