jmbannon / ytdl-sub

Lightweight tool to automate downloading and metadata generation with yt-dlp
https://ytdl-sub.readthedocs.io
GNU General Public License v3.0
1.86k stars 71 forks source link

Errors trying to download full channel #1121

Open Pearcenator opened 1 week ago

Pearcenator commented 1 week ago

I'm getting an error while trying to download the Kurzgesagt channel. Log attached. 20241109T185650.zip

[ytdl-sub:downloader] Downloading metadata for What Are You?
[ytdl-sub:downloader] Beginning downloads for https://www.youtube.com/@kurzgesagt
[ytdl-sub:downloader] Downloading entry 1/2: What Are You?
[ytdl-sub] The following error occurred for the subscription Kurzgesagt – In a Nutshell:
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/ytdl_sub/subscriptions/subscription_download.py", line 388, in exception_handling
    yield
  File "/usr/local/lib/python3.10/dist-packages/ytdl_sub/cli/entrypoint.py", line 126, in _download_subscriptions_from_yaml_files
    subscription.download(dry_run=dry_run)
  File "/usr/local/lib/python3.10/dist-packages/ytdl_sub/subscriptions/subscription_download.py", line 375, in download
    return self._process_subscription(
  File "/usr/local/lib/python3.10/dist-packages/ytdl_sub/subscriptions/subscription_download.py", line 315, in _process_subscription
    entry = downloader.download(entry)
  File "/usr/local/lib/python3.10/dist-packages/ytdl_sub/downloaders/url/downloader.py", line 532, in download
    download_entry = self._extract_entry_info_with_retry(entry=entry)
  File "/usr/local/lib/python3.10/dist-packages/ytdl_sub/downloaders/url/downloader.py", line 352, in _extract_entry_info_with_retry
    download_entry_dict = YTDLP.extract_info_with_retry(
  File "/usr/local/lib/python3.10/dist-packages/ytdl_sub/downloaders/ytdlp.py", line 137, in extract_info_with_retry
    raise FileNotDownloadedException(
ytdl_sub.utils.exceptions.FileNotDownloadedException: yt-dlp failed to download an entry with these arguments: {'ytdl_options': {'ignoreerrors': True, 'outtmpl': '.ytdl-sub-working-directory/Kurzgesagt – In a Nutshell/%(id)S.%(ext)s', 'writethumbnail': True, 'ffmpeg_location': '/usr/bin/ffmpeg', 'download_archive': '.ytdl-sub-working-directory/Kurzgesagt – In a Nutshell/.ytdl-sub-Kurzgesagt – In a Nutshell-download-archive.json-ytdl-archive', 'merge_output_format': 'mp4', 'postprocessors': [{'key': 'FFmpegVideoRemuxer', 'when': 'post_process', 'preferedformat': 'mp4'}, {'key': 'FFmpegMetadata', 'add_chapters': True, 'add_metadata': False}], 'format': "(bv*[ext=mp4][vcodec~='^((he|a)vc|h26[45])']+ba[ext=m4a]) / (bv[ext=mp4]*+ba[ext=m4a]/b)", 'break_on_existing': False, 'max_downloads': 20, 'playlistreverse': True, 'cookiefile': '/config/cookie.txt'}, 'kwargs': {'url': 'https://www.youtube.com/watch?v=JQVmkDUkZT4'}}
[ytdl-sub] Beginning subscription download for M83
[ytdl-sub:downloader] Beginning downloads for https://www.youtube.com/playlist?list=PL7pcMoLbG_DIGNz_6jEZhH6Hll0RT2qm4
[ytdl-sub:downloader] Beginning downloads for https://www.youtube.com/playlist?list=PL7pcMoLbG_DIGNz_6jEZhH6Hll0RT2qm4
[ytdl-sub] 
No files changed for BBC News
[ytdl-sub] 
No files changed for THV 11
[ytdl-sub] 
No files changed for Kurzgesagt – In a Nutshell
[ytdl-sub] 
No files changed for M83
[ytdl-sub] Download Summary:
BBC News                       0 0 0     16 ✔
THV 11                         0 0 0     22 ✔
Kurzgesagt – In a Nutshell     0 0 0     18 FileNotDownloadedException
M83                            0 0 0      2 ✔
Total: 4                       0 0 0     58 Error

See `/tmp/ytdl-sub.errorsxwxjz7ll` for details on errors.
Consider making a GitHub issue including the uploaded log file.

My subscriptions.yaml looks like this:

# __preset__ is a place to define global overrides for all subscriptions
__preset__:
  overrides:
    # Root folder of all ytdl-sub TV Shows
    tv_show_directory: "/tv_shows"

    # Root folder of all ytdl-sub Music
    music_directory: "/music"

    # Root folder of all ytdl-sub Music Videos
    music_video_directory: "/music_videos"

    # For 'Only Recent' preset, only keep vids within this range and limit
    only_recent_date_range: "2days"
    only_recent_max_files: 30

    # For new channel downloads only. Downloads chunks at a time to avoid errors.
    chunk_max_downloads: 20

    # default filter lists to be empty
    title_exclude_keywords: 
        - "#short"
        - "#Short"
    description_exclude_keywords: 
        - "#short"
        - "#Short"

  # Pass any arg directly to yt-dlp's Python API
  ytdl_options:
    cookiefile: "/config/cookie.txt"

####################################################################################################
# Subscriptions nested under this will use the
# `Plex TV Show by Date` preset.
####################################################################################################
Plex TV Show by Date:

  # Set genre tag to "News", use `Only Recent` preset to only store videos uploaded recently
  = News | Only Recent:
    "BBC News": "https://www.youtube.com/@BBCNews"

  = Education | Chunk Downloads:
    "Kurzgesagt – In a Nutshell": "https://www.youtube.com/@kurzgesagt"

####################################################################################################
# Music videos
####################################################################################################
"Plex Music Videos":

  = Dream Pop:
    # Prefixing with '+' puts the subscription into 'map-mode'.
    # Music video presets in map-mode support grouping videos into different
    # categories, which get set on the album field.
    #
    # URLs can either be strings, or maps that can overload title, year, date
    "+ M83":
      #Music Videos:
        #- "https://www.youtube.com/playlist?list=PLOTK54q5K4INNXaHKtmXYr6J7CajWjqeJ"
      Concerts:
         - "https://www.youtube.com/playlist?list=PL7pcMoLbG_DIGNz_6jEZhH6Hll0RT2qm4"
ThomasHaz commented 12 hours ago

Have you tried running with debug logging enabled? (--log-level debug)

I had this same error yesterday when setting up for the first time. I turned on debug logging (saving stdout to file):

ERROR: [youtube] 3N5kWzVtq_Q: Sign in to confirm you’re not a bot. This helps protect our community. Learn more

The issue was that YouTube wanted to ensure I wasn't a bot. After loading a cookie, I was able to successfully download videos.

jmbannon commented 12 hours ago

This can also be a sign of throttle, https://ytdl-sub.readthedocs.io/en/latest/config_reference/plugins.html#throttle-protection can help