ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.34k stars 10.03k forks source link

Support for #EXT-X-BYTERANGE in native hls extractor #10955

Open TehBotolSosro opened 8 years ago

TehBotolSosro commented 8 years ago

Please follow the guide below


Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.10.16. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.


The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue


If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

youtube-dl -v https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/bad/bad.m3u8
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/bad/bad.m3u8']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2016.10.16
[debug] Python version 3.4.4 - Windows-10-10.0.10240
[debug] exe versions: ffmpeg N-82003-g9082603, ffprobe N-82003-g9082603
[debug] Proxy map: {}
[generic] bad: Requesting header
[generic] bad: Downloading m3u8 information
[debug] Invoking downloader on 'https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/bad/bad.m3u8'
[download] Destination: bad-bad.mp4
[debug] ffmpeg command line: ffmpeg -y -headers 'Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/47.0 (Chrome)
Accept-Language: en-us,en;q=0.5
' -i https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/bad/bad.m3u8 -c copy -f mp4 -bsf:a aac_adtstoasc file:bad-bad.mp4.part
ffmpeg version N-82003-g9082603 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.4.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib
  libavutil      55. 32.100 / 55. 32.100
  libavcodec     57. 61.103 / 57. 61.103
  libavformat    57. 52.100 / 57. 52.100
  libavdevice    57.  0.102 / 57.  0.102
  libavfilter     6. 64.100 /  6. 64.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  2.100 /  2.  2.100
  libpostproc    54.  0.100 / 54.  0.100
[NULL @ 0000000002488620] non-existing SPS 0 referenced in buffering period
[NULL @ 0000000002488620] SPS unavailable in decode_picture_timing
[h264 @ 000000000248a2a0] non-existing SPS 0 referenced in buffering period
[h264 @ 000000000248a2a0] SPS unavailable in decode_picture_timing
Input #0, hls,applehttp, from 'https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/bad/bad.m3u8':
  Duration: 00:00:29.95, start: 1.407333, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 90k tbn, 47.95 tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:bad-bad.mp4.part':
  Metadata:
    encoder         : Lavf57.52.100
    Stream #0:0: Video: h264 (Main) ([33][0][0][0] / 0x0021), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 23.98 fps, 23.98 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 48000 Hz, stereo
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=  270 fps=3.7 q=-1.0 Lsize=    4835kB time=00:00:22.50 bitrate=1759.8kbits/s speed=0.308x
video:4656kB audio:173kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.110669%
[ffmpeg] Downloaded 4950862 bytes
[download] 100% of 4.72MiB
...
<end of log>

m3u8 enc https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/encryption/bad/bad.m3u8 unencrypted video : https://s3.amazonaws.com/1d4326f61a9a4ed596de9e1a41d48413/input.ts

Description of your issue, suggested solution and other information

Can you add support for encrypted single file hls (m3u8) in the native hls extractor, since ffmpeg seem have bug reading the byte range resulted in corrupted / skipped video

https://tools.ietf.org/html/draft-pantos-http-live-streaming-20#section-4.3.2.2 https://developer.apple.com/library/content/technotes/tn2288/_index.html#//apple_ref/doc/uid/DTS40012238-CH1-BYTE_RANGE_ https://trac.ffmpeg.org/ticket/5858

also please add support for local m3u8 like youtube-dl.exe "C:\file.m3u8" since sometime it's much faster downloading the encrypted file locally first (to avoid dropped frames)

thanks

yan12125 commented 8 years ago

That's already supported. Install python-crypto and run again.

TehBotolSosro commented 8 years ago

hi thanks for replying, but it still corrupt the video and in other site that use similar technique youtube-dl say "WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg" even after installing python-crypto in windows and add them to PATH

did i do something wrong or youtube-dl is also needed to be added to PATH?

other site m3u8

youtube-dl --hls-prefer-native http:// [generic] hlsc_whe2931_7: Requesting header [generic] hlsc_whe2931_7: Downloading m3u8 information [hlsnative] Downloading m3u8 manifest WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg [download] Destination: hlsc_whe2931_7-hlsc_whe2931_7.mp4 ffmpeg version N-82003-g9082603 Copyright (c) 2000-2016 the FFmpeg developers built with gcc 5.4.0 (GCC) configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-dxva2 --enable-libmfx --enable-nvenc --enable-avisynth --enable-bzlib --enable-libebur128 --enable-fontconfig --enable-frei0r --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librtmp --enable-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-lzma --enable-decklink --enable-zlib libavutil 55. 32.100 / 55. 32.100 libavcodec 57. 61.103 / 57. 61.103 libavformat 57. 52.100 / 57. 52.100 libavdevice 57. 0.102 / 57. 0.102 libavfilter 6. 64.100 / 6. 64.100 libswscale 4. 1.100 / 4. 1.100 libswresample 2. 2.100 / 2. 2.100 libpostproc 54. 0.100 / 54. 0.100 Input #0, hls,applehttp, from 'http://': Duration: 01:29:17.71, start: 0.000000, bitrate: 0 kb/s Program 0 Metadata: variant_bitrate : 0 Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 24 fps, 24 tbr, 90k tbn, 48 tbc Metadata: variant_bitrate : 0 Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp Metadata: variant_bitrate : 0 Output #0, mp4, to 'file:hlsc_whe2931_7-hlsc_whe2931_7.mp4.part': Metadata: encoder : Lavf57.52.100 Stream #0:0: Video: h264 (High) ([33][0][0][0] / 0x0021), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 24 tbr, 90k tbn, 90k tbc Metadata: variant_bitrate : 0 Stream #0:1: Audio: aac (LC) ([64][0][0][0] / 0x0040), 44100 Hz, stereo Metadata: variant_bitrate : 0 Stream mapping: Stream #0:0 -> #0:0 (copy) Stream #0:1 -> #0:1 (copy) Press [q] to stop, [?] for help [hls,applehttp @ 00000000022b2ac0] Failed to open segment of playlist 02.0kbits/s speed=2.57x Last message repeated 1 times [https @ 00000000022bbee0] HTTP error 416 Requested Range Not Satisfiable [crypto @ 00000000022bbee0] Unable to open resource: http://vuclip-a.akamaihd.net/9d34f5174054fba5e979a7d16a84b065/vp63207/9d34_e2931_7.vp.mov.ts [hls,applehttp @ 00000000022b2ac0] Failed to open segment of playlist 0 [crypto @ 00000000022bbee0] Unable to open resource: http://vuclip-a.akamaihd.net/9d34f5174054fba5e979a7d16a84b065/vp63207/9d34_e2931_7.vp.mov.ts [hls,applehttp @ 00000000022b2ac0] Failed to open segment of playlist 0 [crypto @ 00000000022bbee0] Unable to open resource: http://vuclip-a.akamaihd.net/9d34f5174054fba5e979a7d16a84b065/vp63207/9d34_e2931_7.vp.mov.ts [hls,applehttp @ 00000000022b2ac0] Failed to open segment of playlist 0 ---TRUNCATED---

result only the unencrypted video being downloaded while the encrypted video are not downloaded thanks