Live stream VODs within date range from channel with old active live stream

[krafs]

I am trying to download all live stream VODs from Linus Tech Tips from the last two weeks, but an old, always-active live stream gatekeeps them.

Basically: LTT has a continuous live stream that started July 6th. Live streams always appear at the top of the channel's Live playlist, regardless of start date. ytdl-sub downloads the metadata of the live stream, rejects it, because its start date is outside the two-week date range, and breaks. But there are plenty of live stream VODs that are within the two-week date range, but they come after the live stream in the playlist.

Here is my setup, all located and executed from one and the same folder:

config.yaml

configuration:
  working_directory: '/tmp/ytdl-sub'
presets: {}

subscriptions.yaml

ltt:

  preset:
    - 'kodi_tv_show_by_date'
    - 'season_by_year__episode_by_month_day'

  match_filters:
    filters:
      - '!is_live'

  ytdl_options:
    break_on_reject: true

  date_range:
    after: 'today-2weeks'

  overrides:
    tv_show_name: 'LTT'
    tv_show_directory: '/config'
    url: 'https://www.youtube.com/@LinusTechTips/streams'

docker-compose.yaml

services:
  ytdl-sub:
    image: ghcr.io/jmbannon/ytdl-sub:latest
    volumes:
      - ./:/config
    command: ytdl-sub --config=/config/config.yaml --log-level debug sub /config/subscriptions.yaml

I am trying to have ytdl-sub ignore the live stream using match_filters, but ytdl-sub is still inspecting the date range of the video and breaking if outside the range, so it doesn't have the effect I'd hoped.

Thoughts:

• Are date range checks and rejections supposed to run even if a match filter fails?
• Is there a way to have the playlist sorted by date before going through the videos? Since active live streams always are promoted to the start of the playlist, regardless of start date, maybe there is a way to sort it properly so the live stream gets its "real" index based on start date? Then ytdl-sub should work as normal.

[jmbannon]

It's tricky because both match_filters and date_range share:

 ytdl_options:
    break_on_reject: true

Ideally they should have their own breaks - it's a yt-dlp limitation.

There is way to get around this issue though. What you can do is add the following:

  match_filters:
    filters:
      - '!is_live'

  ytdl_options:
    break_on_reject: false
    max_downloads: 100

This will

1. Ensure you do not download live things ('!is_live')
2. Not break immediately (break_on_reject: false)
3. Only pull a max of 100 metadata items (max_downloads: 100) a. If it's already downloaded, it won't be redownloaded because the TV show presets use a download archive

Try that and report back if it works :)

[krafs]

So, I've tried some stuff out:

I used max_downloads: 10 and break_on_reject: false for testing, and it seems like a metadata item download only counts towards max_downloads if the metadata download passes the date range check. Does that sound right? So ytdl-sub will never reach max_downloads because only the first few downloads that fall within the 2 week date range count. After that they stop counting, and it will just proceed with downloading all the metadata items (>300).

Removing the date range makes it work as expected - only 10 will be downloaded on the first go, and then only up until the last. However, this means I wouldn't be able to take advantage of the automatic removal of old videos, right?

[jmbannon]

Interesting! I think what you said is right - glad you found a work around. You can still use the auto deletion without date_range: https://ytdl-sub.readthedocs.io/en/latest/config.html#ytdl_sub.config.preset_options.OutputOptions.keep_files_before

You actually need to add it to your config as:

output_options:
  keep_files_after: "today-2weeks"

[jmbannon]

I'm going to add this feature at some point which may also be a good alternative https://github.com/jmbannon/ytdl-sub/issues/672

[krafs]

I can certainly see a use case for a keep_max_entries. Sounds like a good addition.

But yes - keep_files_after seems to work the way I want for now. Perfect! Many thanks for the help, and good look on the project! I'll be following along :)