ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.04k stars 10.01k forks source link

[FEATURE] exclude segments from HLS playlists #28159

Open mukarator opened 3 years ago

mukarator commented 3 years ago

Checklist

Description

I came across several sites I downloaded some videos from, where the first segment in the playlist (m3u8) is advertising, age restriction or copyright warning.

example: https://puhutv.com/cukur-115-bolum-izle or any other video from the site

1) download the video (lowest quality just for the test):

C:\>youtube-dl -v -c --no-check-certificate -f hls-395-0 https://puhutv.com/cukur-115-bolum-izle
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-c', '--no-check-certificate', '-f', 'hls-395-0', 'https://puhutv.com/cukur-115-bolum-izle']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2021.02.10
[debug] Python version 3.4.4 (CPython) - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg 2021-01-09-git-2e2891383e-full_build-www.gyan.dev, ffprobe 2021-01-09-git-2e2891383e-full_build-www.gyan.dev
[debug] Proxy map: {}
[puhutv] cukur-115-bolum: Downloading JSON metadata
[puhutv] cukur-115-bolum: Downloading video JSON
[puhutv] 23958: Downloading m3u8 information
[puhutv] 23958: Downloading m3u8 information
[puhutv] 23958: Downloading m3u8 information
[puhutv] 23958: Downloading m3u8 information
[puhutv] 23958: Downloading m3u8 information
[puhutv] 23958: Downloading m3u8 information
[debug] Invoking downloader on 'http://ck8di1zz.rocketcdn.com/2021/02/09/62319_6cf726d97d248ef73d285d3ab173a872_max_2160p.smil/62319__6cf726d97d248ef73d285d3ab173a872__P240.mp4_chunk.m3u8?r=6'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 1495
[download] Destination: 115. Bölüm 4. Sezon 115. Bölüm-23958.mp4
[download] 100% of 499.26MiB in 11:12
[debug] ffmpeg command line: ffprobe -show_streams "file:115. Bölüm 4. Sezon 115. Bölüm-23958.mp4"
[ffmpeg] Fixing malformed AAC bitstream in "115. Bölüm 4. Sezon 115. Bölüm-23958.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:115. Bölüm 4. Sezon 115. Bölüm-23958.mp4" -c copy -f mp4 "-bsf:a" aac_adtstoasc "file:115. Bölüm 4. Sezon 115. Bölüm-23958.temp.mp4"

2) copy & paste in the browser the m3u8 link from the ytdl output to download and inspect the playlist :

#EXTM3U
#EXT-X-VERSION:3
#EXT-X-TARGETDURATION:7
#EXT-X-MEDIA-SEQUENCE:0
#EXTINF:5.944,
/rtuk_6_360.ts
#EXT-X-DISCONTINUITY
#EXTINF:6.006,
62319__6cf726d97d248ef73d285d3ab173a872__P240.mp4-n_0_0_0.ts
#EXTINF:6.006,
62319__6cf726d97d248ef73d285d3ab173a872__P240.mp4-n_1_0_0.ts
#EXTINF:6.006,
62319__6cf726d97d248ef73d285d3ab173a872__P240.mp4-n_2_0_0.ts
...
etc.

rtuk_6_360.ts contains the age restriction warning that shows in the beginning of the video, that I want to exclude and download only the main movie. It is even encoded with different parameters than the rest of the video.

My proposal is for option to filter out such unwanted segments from a m3u8 or dash playlist (just like individual videos can be skipped from ordinary playlist with --playlist-start, --playlist-end and --playlist-items options). e.g. something in this manner:

--segment-start, --segment-end, --segment-items

Note my request for this feauture is as general option and not for these specific sites only as I think it will be more flexible if it can be used with the generic extractor as well.

mukarator commented 3 years ago

@dstftw I made some changes to the description, I hope it meets all requirements now

dirkf commented 6 months ago

Continuation of #15748.