HIDIVE download fail - m3u8 "Invalid argument"

JosueLMC commented 4 years ago

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2020.07.28
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-u', u'PRIVATE', u'-p', u'PRIVATE', u'-v', u'--write-thumbnail', u'-a', u'batch.txt']
[debug] Batch file urls: [u'https://www.hidive.com/stream/flip-flappers/s01e009', u'https://www.hidive.com/stream/flip-flappers/s01e010', u'https://www.hidive.com/stream/flip-flappers/s01e011', u'https://www.hidive.com/stream/flip-flappers/s01e012', u'https://www.hidive.com/stream/flip-flappers/s01e013']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 2.7.18rc1 (CPython) - Linux-5.4.0-7634-generic-x86_64-with-Pop-20.04-focal
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[HiDive] None: Downloading webpage
[HiDive] Logging in
[HiDive] flip-flappers/s01e009: Downloading JSON metadata
[HiDive] flip-flappers/s01e009: Downloading m3u8 information
[HiDive] flip-flappers/s01e009: Downloading m3u8 information
[HiDive] flip-flappers/s01e009: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://www.hidive.com/manifest/child/2601bcbe35629087ad665fd775991e72ac6e5514/FLP_s01e009_tv_hv_or_en_xx_HLS1080p_16x9_00_v.m3u8?#='
[hlsnative] Downloading m3u8 manifest
WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg
[download] Destination: flip-flappers_s01e009-flip-flappers_s01e009.mp4
[debug] ffmpeg command line: ffmpeg -y -loglevel verbose -headers 'Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3564.0 Safari/537.36
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Cookie: 'PRIVATE' -i 'https://www.hidive.com/manifest/child/2601bcbe35629087ad665fd775991e72ac6e5514/FLP_s01e009_tv_hv_or_en_xx_HLS1080p_16x9_00_v.m3u8?#=' -c copy -f mp4 'file:flip-flappers_s01e009-flip-flappers_s01e009.mp4.part'
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
  configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[tcp @ 0x55b697903fc0] Starting connection attempt to 52.25.229.103 port 443
[tcp @ 0x55b697903fc0] Successfully connected to 52.25.229.103 port 443
[https @ 0x55b697903400] overlong headers
https://www.hidive.com/manifest/child/2601bcbe35629087ad665fd775991e72ac6e5514/FLP_s01e009_tv_hv_or_en_xx_HLS1080p_16x9_00_v.m3u8?#=: Invalid argument

ERROR: ffmpeg exited with code 1
  File "/usr/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/bin/youtube-dl/__main__.py", line 19, in <module>
    youtube_dl.main()
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 474, in main
    _real_main(argv)
  File "/usr/local/bin/youtube-dl/youtube_dl/__init__.py", line 464, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2019, in download
    url, force_generic_extractor=self.params.get('force_generic_extractor', False))
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 808, in extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 863, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1644, in process_video_result
    self.process_info(new_info)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1926, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1865, in dl
    return fd.download(name, info)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/hls.py", line 76, in real_download
    return fd.real_download(filename, info_dict)
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/external.py", line 64, in real_download
    self.get_basename(), retval))
  File "/usr/local/bin/youtube-dl/youtube_dl/downloader/common.py", line 165, in report_error
    self.ydl.report_error(*args, **kargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 625, in report_error
    self.trouble(error_message, tb)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 587, in trouble
    tb_data = traceback.format_list(traceback.extract_stack())

Description

Hi, youtube-dl fails when download a episode from HIDIVE, the login works fine, recognise the files, but when start the stream pop me the error.

digitalw00t commented 3 years ago

I'm seeing this error now as well. Has there been a fix for this, or a cause?

ghost commented 3 years ago

The error is of ffmpeg:

[https @ 0x55b697903400] overlong headers

There is a limit in ffmpeg for max header/buffer length. But reading from this warning message,

WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg

you may let youtube-dl to do actual download if you have pycryptodome module installed and it is not a live stream.

If that fails still, try with newer version of ffmpeg 4.3+. If that even fails, you may need to change ffmpeg source code and rebuild, like https://github.com/ytdl-org/youtube-dl/pull/24649#issuecomment-804811788.

ytdl-org / youtube-dl