search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.6k stars 10.05k forks source link

Add site support for banned.video #22718

Open ghost opened 5 years ago

ghost commented 5 years ago

Checklist

Example URLs

Description

Only video works after many fallbacks, no audio, takes long to init. Please also support automatic title to filename.

The single videos all correspond to the same video. Livestream links correspond to another video. "Channel Livestreams" don't work at all, unsupported URL.

Additional domains: Player: https://www.infowarsmedia.com/js/player.js Player2: https://cdn.irsdn.net/videojs-hlsjs-plugin/1/stable/videojs-hlsjs-plugin.js Referr: https://api.infowarsmedia.com/embed/5da4afaf9f040f0014733e18 HLS: https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/manifest/stream_3.m3u8 There's also an "incremental" url but doesn't work by it self.

(OperaBrowser) Right Click -> Copy Video Address seems to throw out an ever changing blob API link using same domain as the embed/referr domain, but the guid is always different when copying and also always invalid, it won't connect. doesn't seem like it's important, but I mentioned it anyway. blob:https://api.infowarsmedia.com/14d3fa9f-dc4b-4f81-a29f-d25fb7f3ab39 and sometimes another version: blob:https://vod-api.infowars.com/40a4552c-20e0-44f3-bd2e-506ac1b1c17c

Livestreams don't seem to have a special video link, they're inserted into the banned.video channel page. Maybe there is one, but the API and Blob ones I tried again return 404 or have errors (probably some kind of security system maybe, or metadata only accessible by scripts)

Channel Livestream M3U8: https://infostream.secure.footprint.net/hls-live/infostream3-infostream3/_definst_/live.m3u8

StaticDL: https://api.infowarsmedia.com/api/video/5da4afaf9f040f0014733e18/download StaticDL Direct: https://assets.infowarsmedia.com/videos/2892eae3-88b9-4f26-a56a-ac8252cd3238.mov (for some reason another GUID here but corresponds to 5da4a..)

Static MOV download is the highest quality and separate of HLS VOD and Livestream, this works ofcourse out of the box as it's meant for downloading, but it's not the most optimal choice when it comes to storage space so.

Verbose Output

C:\Program Files Manual\YoutubeDL>youtube-dl -F https://banned.video/watch?id=5da4afaf9f040f0014733e18 -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-F', 'https://banned.video/watch?id=5da4afaf9f040f0014733e18', '-v']
[debug] Encodings: locale cp1250, fs mbcs, out cp852, pref cp1250
[debug] youtube-dl version 2019.09.28
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-94664-g0821bc4eee, ffprobe N-94664-g0821bc4eee
[debug] Proxy map: {}
[generic] watch?id=5da4afaf9f040f0014733e18: Requesting header
WARNING: Falling back on generic information extractor.
[generic] watch?id=5da4afaf9f040f0014733e18: Downloading webpage
[generic] watch?id=5da4afaf9f040f0014733e18: Extracting information
[generic] 5da4afaf9f040f0014733e18: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 5da4afaf9f040f0014733e18: Downloading webpage
[generic] 5da4afaf9f040f0014733e18: Extracting information
[generic] 5da4afaf9f040f0014733e18: Downloading m3u8 information
[download] Downloading playlist: video
[generic] playlist video: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[info] Available formats for 5da4afaf9f040f0014733e18:
format code              extension  resolution note
hls-group_audio-audio_0  mp4        audio only
hls-580                  mp4        426x240     580k , avc1.42c015, video only
hls-1020                 mp4        640x360    1020k , avc1.4d401e, video only
hls-2120                 mp4        854x480    2120k , avc1.4d401f, video only
hls-4100                 mp4        1280x720   4100k , avc1.4d401f, video only
hls-5860                 mp4        1920x1080  5860k , avc1.4d4028, video only (best)
[download] Finished downloading playlist: video

C:\Program Files Manual\YoutubeDL>youtube-dl -f hls-4100 https://banned.video/watch?id=5da4afaf9f040f0014733e18 -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'hls-4100', 'https://banned.video/watch?id=5da4afaf9f040f0014733e18', '-v']
[debug] Encodings: locale cp1250, fs mbcs, out cp852, pref cp1250
[debug] youtube-dl version 2019.09.28
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-94664-g0821bc4eee, ffprobe N-94664-g0821bc4eee
[debug] Proxy map: {}
[generic] watch?id=5da4afaf9f040f0014733e18: Requesting header
WARNING: Falling back on generic information extractor.
[generic] watch?id=5da4afaf9f040f0014733e18: Downloading webpage
[generic] watch?id=5da4afaf9f040f0014733e18: Extracting information
[generic] 5da4afaf9f040f0014733e18: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 5da4afaf9f040f0014733e18: Downloading webpage
[generic] 5da4afaf9f040f0014733e18: Extracting information
[generic] 5da4afaf9f040f0014733e18: Downloading m3u8 information
[download] Downloading playlist: video
[generic] playlist video: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[debug] Invoking downloader on 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/manifest/stream_4.m3u8'
[download] Destination: video-5da4afaf9f040f0014733e18.mp4
[debug] ffmpeg command line: ffmpeg -y -loglevel verbose -headers "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3738.4 Safari/537.36
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Accept-Encoding: gzip, deflate
Accept-Language: en-us,en;q=0.5
Cookie: __cfduid=d6680d978e00d9fdf34688d4027c01ab41571117218
Referer: https://api.infowarsmedia.com/embed/5da4afaf9f040f0014733e18
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
" -i "https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/manifest/stream_4.m3u8" -c copy -f mp4 "file:video-5da4afaf9f040f0014733e18.mp4.part"
ffmpeg version N-94664-g0821bc4eee Copyright (c) 2000-2019 the FFmpeg developers
  built with gcc 9.1.1 (GCC) 20190807
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt
  libavutil      56. 33.100 / 56. 33.100
  libavcodec     58. 55.101 / 58. 55.101
  libavformat    58. 31.104 / 58. 31.104
  libavdevice    58.  9.100 / 58.  9.100
  libavfilter     7. 58.101 /  7. 58.101
  libswscale      5.  6.100 /  5.  6.100
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
[tcp @ 000001938b55c880] Starting connection attempt to 104.17.106.42 port 443
[tcp @ 000001938b55c880] Successfully connected to 104.17.106.42 port 443
[hls @ 000001938b559600] Skip ('#EXT-X-VERSION:6')
[hls @ 000001938b559600] Skip ('#EXT-X-INDEPENDENT-SEGMENTS')
[hls @ 000001938b559600] Skip ('#Stream job=ApPJdf5x4qI= try=998095')
[hls @ 000001938b559600] HLS request for url 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_0.ts', offset 0, playlist 0
[hls @ 000001938b559600] Opening 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_0.ts' for reading
[tcp @ 000001938bc21d40] Starting connection attempt to 104.17.106.42 port 443
[tcp @ 000001938bc21d40] Successfully connected to 104.17.106.42 port 443
[hls @ 000001938b559600] HLS request for url 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_1.ts', offset 0, playlist 0
[hls @ 000001938b559600] Opening 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_1.ts' for reading
[tcp @ 000001938bc109c0] Starting connection attempt to 104.17.106.42 port 443
[tcp @ 000001938bc109c0] Successfully connected to 104.17.106.42 port 443
[h264 @ 000001938bfc8fc0] Reinit context to 1280x720, pix_fmt: yuv420p
[hls @ 000001938b559600] HLS request for url 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_2.ts', offset 0, playlist 0
[https @ 000001938bc0aa80] Opening 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_2.ts' for reading
Input #0, hls, from 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/manifest/stream_4.m3u8':
  Duration: 00:19:50.83, start: 0.066667, bitrate: 0 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (Main), 1 reference frame ([27][0][0][0] / 0x001B), yuv420p(left), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:video-5da4afaf9f040f0014733e18.mp4.part':
  Metadata:
    encoder         : Lavf58.31.104
    Stream #0:0: Video: h264 (Main), 1 reference frame (avc1 / 0x31637661), yuv420p(left), 1280x720 (0x0) [SAR 1:1 DAR 16:9], q=2-31, 30 fps, 30 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[hls @ 000001938b559600] HLS request for url 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_3.ts', offset 0, playlist 0
[https @ 000001938bda3040] Opening 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_3.ts' for reading
[hls @ 000001938b559600] HLS request for url 'https://bytehighway.net/7a99a5cbb9409bcb49b7ef8a3df84683/video/720/stream_4-seg_4.ts', offset 0, playlist 0
shillshocked commented 4 years ago

+1, I would love this.

blackdimund commented 4 years ago

+1

RankoKohime commented 4 years ago

+1

rmlrml commented 3 years ago

+1

lee321987 commented 3 years ago

+1

bill-mcgonigle commented 2 years ago

There's support for Cloudflare Stream/Bytehighway in another module so this might be more straightforward now.

hsn10 commented 2 years ago

Currently, it fails on sending HEAD request.

youtube-dl --verbose https://banned.video/watch?id=62e4a96029060f37494c4f2a [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--verbose', 'https://banned.video/watch?id=62e4a96029060f37494c4f2a'] [debug] Encodings: locale cp1250, fs mbcs, out cp852, pref cp1250 [debug] youtube-dl version 2021.12.17 [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041 [debug] exe versions: ffmpeg 5.1-full_build-www.gyan.dev, ffprobe 5.1-full_build-www.gyan.dev [debug] Proxy map: {} [generic] watch?id=62e4a96029060f37494c4f2a: Requesting header WARNING: Could not send HEAD request to https://banned.video/watch?id=62e4a96029060f37494c4f2a: HTTP Error 403: Forbidden [generic] watch?id=62e4a96029060f37494c4f2a: Downloading webpage ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 634, in _request_webpage File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen File "C:\Python\Python34\lib\urllib\request.py", line 470, in open File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response File "C:\Python\Python34\lib\urllib\request.py", line 508, in error File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

hsn10 commented 2 years ago

https://github.com/yt-dlp/yt-dlp/blob/66cf3e1001b6d9a2829fe834c3f9103b0890918e/yt_dlp/extractor/bannedvideo.py