Open remitamine opened 8 years ago
(For reference, this uses the url posted in #7066)
This only happens for HEAD requests because it returns no data, therefore zlib can't decompress it. This simple patch "fixes" it:
diff --git a/youtube_dl/utils.py b/youtube_dl/utils.py
index 7dbe256..dee3b00 100644
--- a/youtube_dl/utils.py
+++ b/youtube_dl/utils.py
@@ -670,6 +670,9 @@ class YoutubeDLHandler(compat_urllib_request.HTTPHandler):
@staticmethod
def deflate(data):
+ # HEAD requests produce no data, which is not a valid compressed file
+ if not data:
+ return data
try:
return zlib.decompress(data, -zlib.MAX_WBITS)
except zlib.error:
Although it could be better to handle it in YoutubeDLHandler.http_response.
How do I apply this "fix"? @jaimeMF
@jaimeMF It's been five years, but thank you for this patch. I was just able to fix a failing download by applying your patch to youtube-dl version 2020.11.21.1.
The URL of the failing page is https://www.pointfree.co/episodes/ep100-a-tour-of-the-composable-architecture-part-1
Without the patch:
$ youtube-dl --version
2020.11.21.1
$ youtube-dl -v https://www.pointfree.co/episodes/ep100-a-tour-of-the-composable-architecture-part-1
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.pointfree.co/episodes/ep100-a-tour-of-the-composable-architecture-part-1']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.11.21.1
[debug] Python version 3.9.0 (CPython) - macOS-10.15.7-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[generic] ep100-a-tour-of-the-composable-architecture-part-1: Requesting header
Traceback (most recent call last):
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/utils.py", line 2588, in deflate
return zlib.decompress(data, -zlib.MAX_WBITS)
zlib.error: Error -5 while decompressing data: incomplete or truncated stream
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/bin/youtube-dl", line 33, in <module>
sys.exit(load_entry_point('youtube-dl==2020.11.21.1', 'console_scripts', 'youtube-dl')())
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/__init__.py", line 474, in main
_real_main(argv)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/__init__.py", line 464, in _real_main
retcode = ydl.download(all_urls)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2028, in download
res = self.extract_info(
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
return self.__extract_info(url, ie, download, extra_info, process)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 803, in wrapper
return func(self, *args, **kwargs)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 824, in __extract_info
ie_result = ie.extract(url)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 532, in extract
ie_result = self._real_extract(url)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/extractor/generic.py", line 2296, in _real_extract
head_response = self._request_webpage(
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/extractor/common.py", line 632, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2248, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/local/Cellar/python@3.9/3.9.0_2/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 523, in open
response = meth(req, response)
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/utils.py", line 2648, in http_response
gz = io.BytesIO(self.deflate(resp.read()))
File "/usr/local/Cellar/youtube-dl/2020.11.21.1_1/libexec/lib/python3.9/site-packages/youtube_dl/utils.py", line 2590, in deflate
return zlib.decompress(data)
zlib.error: Error -5 while decompressing data: incomplete or truncated stream
With the patch:
youtube-dl https://www.pointfree.co/episodes/ep100-a-tour-of-the-composable-architecture-part-1
[generic] ep100-a-tour-of-the-composable-architecture-part-1: Requesting header
WARNING: Could not send HEAD request to https://www.pointfree.co/episodes/ep100-a-tour-of-the-composable-architecture-part-1: HTTP Error 404: Not Found
[generic] ep100-a-tour-of-the-composable-architecture-part-1: Downloading webpage
WARNING: Falling back on generic information extractor.
[generic] ep100-a-tour-of-the-composable-architecture-part-1: Extracting information
[download] Downloading playlist: Episode #100: A Tour of the Composable Architecture: Part 1
[generic] playlist Episode #100: A Tour of the Composable Architecture: Part 1: Collected 1 video ids (downloading 1 of them)
[download] Downloading video 1 of 1
[vimeo] 414016119: Downloading webpage
[vimeo] 414016119: Extracting information
[vimeo] 414016119: Downloading JSON metadata
WARNING: Unable to download JSON metadata: HTTP Error 404: Not Found
[vimeo] 414016119: Downloading fastly_skyfire m3u8 information
[vimeo] 414016119: Downloading fastly_skyfire m3u8 information
[vimeo] 414016119: Downloading fastly_skyfire MPD information
[vimeo] 414016119: Downloading fastly_skyfire MPD information
[dashsegments] Total fragments: 327
[download] Destination: 0100-414016119.fdash-fastly_skyfire_sep-video-b4791714.mp4
[download] 4.7% of ~780.34MiB at 6.73MiB/s ETA 02:02^C
It would be great for this patch or something like it to make it into youtube-dl.