search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.89k stars 10k forks source link

dailymotion urlopen error [Errno 1] _ssl.c:504: :sslv3 alert handshake failure #32934

Closed tansy closed 2 weeks ago

tansy commented 2 weeks ago

Checklist

Verbose log


$ youtube-dl -v https://www.dailymotion.com/video/xxsxxf
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'https://www.dailymotion.com/video/xxsxxf']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2024.08.07 [c5098961b] (single file build)
[debug] ** This version was built from the latest master code at https://github.com/ytdl-org/youtube-dl.
[debug] ** For support, visit the main site.
[debug] Python 2.7.3 (CPython i686 32bit) - OpenSSL 1.0.1e 11 Feb 2013 - glibc 2.0
[debug] exe versions: none
[debug] Proxy map: {}
[dailymotion] Downloading Access Token
[dailymotion] xxsxxf: Downloading media JSON metadata
[dailymotion] xxsxxf: Downloading metadata JSON
[dailymotion] xxsxxf: Downloading m3u8 information
[debug] Default format spec: best/bestvideo+bestaudio
[debug] Invoking downloader on u'https://vod.cf.dmcdn.net/sec2(k_HTfzxJEHb4hrFRwBuS308_ZIsyzKmvdkUtMkmlAAooEM14-gN0AJW53TN2lwbFtocEJlWPZSZLkVZ67WSvqlGhMtQjHz4EJmU_hfThvtQl7KcUnGYunkYqeukYmvlBXgYCqGBnqIe9cA2Nkm9an3khreEjIeSFovbIDjrFuWA)/video/766/777/56777667_mp4_h264_aac_hd.mp4'
ERROR: unable to download video data: <urlopen error [Errno 1] _ssl.c:504: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>
Traceback (most recent call last):
  File "/tmp/youtube_dl/YoutubeDL.py", line 2162, in process_info
    success = dl(filename, info_dict)
  File "/tmp/youtube_dl/YoutubeDL.py", line 2092, in dl
    return fd.download(name, new_info)
  File "/tmp/youtube_dl/downloader/common.py", line 380, in download
    return self.real_download(filename, info_dict)
  File "/tmp/youtube_dl/downloader/http.py", line 349, in real_download
    establish_connection()
  File "/tmp/youtube_dl/downloader/http.py", line 116, in establish_connection
    raise err
URLError: <urlopen error [Errno 1] _ssl.c:504: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure>

Description

Any/every dailymotion video fails with the same error message. One can probably figure out what that part in brackets, in the video URL (after vod.cf.dmcdn.net/sec2), actually mean but I couldn't get what that is. Not a javascript inspector boss.

dirkf commented 2 weeks ago

This is with a Miniconda build of Py2.7

$ python -m youtube_dl -v --test 'https://www.dailymotion.com/video/xxsxxf'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'--test', u'https://www.dailymotion.com/video/xxsxxf']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: a4d4bbdb0
[debug] Python 2.7.15 (CPython i686 32bit) - Linux-6.1.0-25-686-pae-i686-with-debian-12.7 - OpenSSL 1.1.1a  20 Nov 2018 - glibc 2.1.3
[debug] exe versions: ffmpeg 5.1.6-0, ffprobe 5.1.6-0
[debug] Proxy map: {}
[dailymotion] Downloading Access Token
[dailymotion] xxsxxf: Downloading media JSON metadata
[dailymotion] xxsxxf: Downloading metadata JSON
[dailymotion] xxsxxf: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on u'https://vod.cf.dmcdn.net/sec2(RtUwwliZ9TC3RuvNWIXciaEKgdIPfhAXe8-kbdO86sBM-jT-e_QtRNZfG7qlgaLg1LhlO9SiNQc6_LdlgkI2DJudGRO2TnRkrVCmYJuLVx-oLdlXHfveaxHhEDeY_0Ui5m6O87UgdSi6-5vAg6i-MZf2EHRglXhlgpO-Snkgk9s)/video/766/777/56777667_mp4_h264_aac_hd.mp4'
[download] Destination: Chelyabinsk Russian Meteor Meteorite Shower Explosion 2-15-2013 [ALL FOOTAGE]-xxsxxf.mp4
[download] 100% of 10.00KiB in 00:00
$

  1. Did this work before (ie with DM)?

  2. Are you able to upgrade to a newer Python 2 build?

My initial guess is that the linked OpenSSL from 2013 is too old for DM.

tansy commented 2 weeks ago

I think you're right, but it turns to be more difficult than anyone would expect. I tried to use newer python and fails to cooperate with youtube-dl altogether

$ python --version
Python 2.7.11

$ ytdl-nightly-2024.08.07 -F https://www.dailymotion.com/video/xxsxxf
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "./ytdl-nightly-2024.08.07-V.pyexe-pyc/__main__.py", line 16, in <module>
  File "/tmp/youtube_dl/__init__.py", line 14, in <module>
  File "/tmp/youtube_dl/options.py", line 8, in <module>
  File "/tmp/youtube_dl/downloader/__init__.py", line 3, in <module>
  File "/tmp/youtube_dl/utils.py", line 30, in <module>
  File "/usr/lib/python2.7/ssl.py", line 97, in <module>
    import _ssl             # if we can't import it, let the error propagate
ImportError: /usr/lib/python2.7/lib-dynload/_ssl.so: undefined symbol: SSL_CTX_set_alpn_protos

though it gave me so clues, namely `_ssl.so: undefined symbol: SSL_CTX_set_alpn_protos'. I compiled newer openssl-1.1.1, checked for that symbol (SSL_CTX_set_alpn_protos), it was there, and with it it crashes, regardless of python version.

$ LD_PRELOAD="libssl.so.1.1 libcrypto.so.1.1" ytdl-nightly-2024.08.07 -F https://www.dailymotion.com/video/xxsxxf
*** glibc detected *** python: double free or corruption (!prev): 0x09e393e8 ***
*** glibc detected *** python: malloc(): memory corruption: 0x09e39450 ***
^C^C^CTerminated

And I compiled both, python and opessl-1.1.1 myself. So I'm in a fix now.

I guess it (the topic) is 'off topic' in general, and I have to figure out what is wrong with these programs.

This is with a Miniconda build of Py2.7 Are you able to upgrade to a newer Python 2 build?

Yes, but it also might be because it has its own OpenSSL package ($miniconda3/lib/python3.7/site-packages/OpenSSL). Me think it's ridiculous to have 400MB container to do that, apparently simple thing. It's the python or libssl.so that should fixed rather than adding another replacement.

My initial guess is that the linked OpenSSL from 2013 is too old for DM.

As above - have to figure out why it crashes, maybe that will help.

dirkf commented 2 weeks ago

The OpenSSL 1.1.1a lib is included in the Miniconda source as you suggest. You could probably bin the main space hog (pkgs directory ~350MB) and anything to do with Py3.7 once the build is done.

dirkf commented 2 weeks ago

Was the original 2.7.3 Python the version from the distro's package manager (and what was that combo)? I want to check that, as I believe, the code is behaving as expected.

The yt-dl SSL code is "designed" to fail back to insecure settings if the settings requested by the other party are not supported: if the other party refuses the downgraded settings, one could get an error like the one shown in the original log.

tansy commented 2 weeks ago

Was the original 2.7.3 Python the version from the distro's package manager (and what was that combo)?

After checking it has to be - yes, it came with distro. It used to be slackware, 14.x, can't remember exactly now as I replaced almost every piece of it over time. As it turns out not every one.

And I compiled both, python (...) myself.

I checked that - it turns out I didn't. Did it again, only just, with v2.7.11, and it doesn't throw errors anymore, along with youtube-dl.

So it's end of the saga. Thanks for help - that container gave me some perspective that help to figure out the problem. Though I still don't know exactly whether it is/was python's fault, or SSL's fault but with time I will find out because that openssl crashing bugs me still. But it's fixed now, and will close the issue.

dirkf commented 2 weeks ago

I'm pretty sure that this explains the original issue:

... if the other party refuses the downgraded settings, one could get an error like the one shown in the original log.