ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.61k stars 9.97k forks source link

setting file modification time throws ValueError: year is out of range #27603

Open mangoducksparkles opened 3 years ago

mangoducksparkles commented 3 years ago

Checklist

Verbose log

$ youtube-dl -v https://www.youtube.com/watch?v=D22BOOGbpFM
[debug] System config: ['-w', '-c', '--write-sub', '--sub-lang', 'en', '--embed-subs', '-f', 'best[width<1280]', '-o', '~/Downloads/%(uploader)s/%(upload_date)s - %(title)s.%(ext)s']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=D22BOOGbpFM']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.12.31
[debug] Python version 3.9.1 (CPython) - macOS-10.14.6-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] D22BOOGbpFM: Downloading webpage
WARNING: en subtitles not available for D22BOOGbpFM
[debug] Invoking downloader on 'https://r1---sn-nx5s7n76.googlevideo.com/videoplayback?expire=1609461428&ei=VBruX_ixBJCJkwacuouoDg&ip=23.19.87.220&id=o-AGWoN-a7N3TgKnwSJQY_0pZe8xocBOEzXT_mwMNZwdZb&itag=18&source=youtube&requiressl=yes&mh=O-&mm=31%2C29&mn=sn-nx5s7n76%2Csn-nx5e6nez&ms=au%2Crdu&mv=m&mvi=1&pl=21&initcwndbps=710000&vprv=1&mime=video%2Fmp4&ns=84XJ9c9UYTps6IbsZWB-4ZEF&gir=yes&clen=8063487&ratebypass=yes&dur=99.938&lmt=3545513993541138233&mt=1609439562&fvip=1&c=WEB&n=75VD4WsBoji0kKuQ&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cvprv%2Cmime%2Cns%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&sig=AOq0QJ8wRQIgIyQboR7HQoVZMoUkgRmnaitKlEfjmqO7RVntJZBHB-UCIQCTWAMdXiIsAeqNuRmdJGVky98sLdT2NHndYPF0mEbLkw%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRQIgcGsoofwf8Uuzrz-oT8MOI3Iz5tAgPSaBVIRMw0YuSKcCIQD8_CCzhdtG2-5jxBsGz4qWCz9yrw42mAkIIsStEyC3zw%3D%3D'
[download] Destination: /Users/username/Downloads/Tom Scott/20150112 - The Magic Roundabout - Swindon's Terrifying Traffic Circle and Emergent Behaviour.mp4
[download] 100.0% of 7.69MiB at  1.30MiB/s ETA 00:00Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl", line 33, in <module>
    sys.exit(load_entry_point('youtube-dl==2020.12.31', 'console_scripts', 'youtube-dl')())
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/__init__.py", line 474, in main
    _real_main(argv)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/__init__.py", line 464, in _real_main
    retcode = ydl.download(all_urls)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 2028, in download
    res = self.extract_info(
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 796, in extract_info
    return self.__extract_info(url, ie, download, extra_info, process)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 803, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 835, in __extract_info
    return self.process_ie_result(ie_result, download, extra_info)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 869, in process_ie_result
    return self.process_video_result(ie_result, download=download)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 1654, in process_video_result
    self.process_info(new_info)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 1936, in process_info
    success = dl(filename, info_dict)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/YoutubeDL.py", line 1875, in dl
    return fd.download(name, info)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/downloader/common.py", line 366, in download
    return self.real_download(filename, info_dict)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/downloader/http.py", line 352, in real_download
    return download()
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/downloader/http.py", line 337, in download
    info_dict['filetime'] = self.try_utime(ctx.filename, ctx.data.info().get('last-modified', None))
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/downloader/common.py", line 215, in try_utime
    filetime = timeconvert(timestr)
  File "/usr/local/Cellar/youtube-dl/2020.12.31/libexec/lib/python3.9/site-packages/youtube_dl/utils.py", line 2075, in timeconvert
    timestamp = email.utils.mktime_tz(timetuple)
  File "/usr/local/Cellar/python@3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/email/_parseaddr.py", line 189, in mktime_tz
    t = calendar.timegm(data)
  File "/usr/local/Cellar/python@3.9/3.9.1_3/Frameworks/Python.framework/Versions/3.9/lib/python3.9/calendar.py", line 658, in timegm
    days = datetime.date(year, month, 1).toordinal() - _EPOCH_ORD + day - 1
ValueError: year 114322 is out of range

Description

Sorry, I'm unsure if this is truly a bug or worth fixing. When I try to download this video, it successfully downloads, but then throws an error and (if part of a playlist) stops the download process.

The file is correctly named with the upload_date (see the -o argument in the System config), and it only fails when it tries to set the file modification time (passing the argument --no-mtime bypasses this issue). Therefore, I'm guessing that the time metadata downloaded from youtube is correct, and that there is a problem with the function that sets the modification time. However, the --no-mtime documentation states that it uses the Last-modified header, so perhaps that is different than the upload_date, and this is just a case of youtube having some bad metadata. Sorry, I don't know.

If this is just the case of bad youtube metadata, I don't know that anything needs to be done. This can be bypassed with --no-mtime, as I mentioned, and I assume that -i would allow the playlist to continue downloading in case this happens again. However, if this is an actual bug with the function that sets the modification time, then that should probably be investigated. I'm sorry, I don't have any suggested solution.

ghost commented 3 years ago

header: Last-Modified: Mon, 13 Nov 114322 14:12:21 GMT

This is shown with --print-traffic. Year 114322 seems to violate RFC specification (= 4 digit).

Since this is not user input data, and probably very rare, I guess no workaround code will be made or needed to prevent exception termination.