mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.38k stars 930 forks source link

[twitter] - SSL/TLS deprication error while running from source files #4766

Open ghbook opened 10 months ago

ghbook commented 10 months ago

Strange,

gives SSL error, while running from source files i.e. if download zip from github and run. gdl is just alias.

NOTE: this SSL error occurs only for twitter, remaining extractors working fine.

D:\gh-gallery-dl>gdl -v https://twitter.com/NASA/status/1720534102934229085
[gallery-dl][debug] Version 1.26.2-dev - Git HEAD: 75ba32af
[gallery-dl][debug] Python 3.11.4 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.29.0 - urllib3 1.26.16
[gallery-dl][debug] Configuration Files ['%APPDATA%\\gallery-dl\\config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://twitter.com/NASA/status/1720534102934229085'
[twitter][debug] Using TwitterTweetExtractor for 'https://twitter.com/NASA/status/1720534102934229085'
Traceback (most recent call last):
  File "D:\gh-gallery-dl\gallery_dl\__main__.py", line 20, in <module>
    gallery_dl.main()
  File "D:\gh-gallery-dl\gallery_dl\__init__.py", line 296, in main
    retval |= jobtype(url).run()
              ^^^^^^^^^^^^^^^^^^
  File "D:\gh-gallery-dl\gallery_dl\job.py", line 116, in run
    self._init()
  File "D:\gh-gallery-dl\gallery_dl\job.py", line 205, in _init
    self.extractor.initialize()
  File "D:\gh-gallery-dl\gallery_dl\extractor\common.py", line 63, in initialize
    self._init_session()
  File "D:\gh-gallery-dl\gallery_dl\extractor\common.py", line 346, in _init_session
    adapter = _build_requests_adapter(
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\gh-gallery-dl\gallery_dl\extractor\common.py", line 778, in _build_requests_adapter
    ssl_context.options |= ssl_options
    ^^^^^^^^^^^^^^^^^^^
  File "C:\Python3\Lib\ssl.py", line 624, in options
    super(SSLContext, SSLContext).options.__set__(self, value)
DeprecationWarning: ssl.OP_NO_SSL*/ssl.OP_NO_TLS* options are deprecated

  twitter working fine if install via pip and use it

D:\>gallery-dl -v https://twitter.com/NASA/status/1720534102934229085
[gallery-dl][debug] Version 1.26.1
[gallery-dl][debug] Python 3.11.4 - Windows-10-10.0.19045-SP0
[gallery-dl][debug] requests 2.29.0 - urllib3 1.26.16
[gallery-dl][debug] Configuration Files ['%APPDATA%\\gallery-dl\\config.json']
[gallery-dl][debug] Starting DownloadJob for 'https://twitter.com/NASA/status/1720534102934229085'
[twitter][debug] Using TwitterTweetExtractor for 'https://twitter.com/NASA/status/1720534102934229085'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): twitter.com:443
[urllib3.connectionpool][debug] https://twitter.com:443 "GET /i/api/graphql/2ICDjqPd81tulZcYrtpTuQ/TweetResultByRestId?variables=%7B%22tweetId%22%3A%221720534102934229085%22%2C%22withCommunity%22%3Afalse%2C%22includePromotedContent%22%3Afalse%2C%22withVoice%22%3Afalse%7D&features=%7B%22creator_subscriptions_tweet_preview_api_enabled%22%3Atrue%2C%22tweetypie_unmention_optimization_enabled%22%3Atrue%2C%22responsive_web_edit_tweet_api_enabled%22%3Atrue%2C%22graphql_is_translatable_rweb_tweet_is_translatable_enabled%22%3Atrue%2C%22view_counts_everywhere_api_enabled%22%3Atrue%2C%22longform_notetweets_consumption_enabled%22%3Atrue%2C%22responsive_web_twitter_article_tweet_consumption_enabled%22%3Afalse%2C%22tweet_awards_web_tipping_enabled%22%3Afalse%2C%22freedom_of_speech_not_reach_fetch_enabled%22%3Atrue%2C%22standardized_nudges_misinfo%22%3Atrue%2C%22tweet_with_visibility_results_prefer_gql_limited_actions_policy_enabled%22%3Atrue%2C%22longform_notetweets_rich_text_read_enabled%22%3Atrue%2C%22longform_notetweets_inline_media_enabled%22%3Atrue%2C%22responsive_web_graphql_exclude_directive_enabled%22%3Atrue%2C%22verified_phone_label_enabled%22%3Afalse%2C%22responsive_web_media_download_video_enabled%22%3Afalse%2C%22responsive_web_graphql_skip_user_profile_image_extensions_enabled%22%3Afalse%2C%22responsive_web_graphql_timeline_navigation_enabled%22%3Atrue%2C%22responsive_web_enhance_cards_enabled%22%3Afalse%7D&fieldToggles=%7B%22withArticleRichContentState%22%3Afalse%7D HTTP/1.1" 200 1757
[twitter][debug] Active postprocessor modules: [MetadataPP]
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): pbs.twimg.com:443
[urllib3.connectionpool][debug] https://pbs.twimg.com:443 "GET /media/F-CQpXiWoAALq4b?format=jpg&name=orig HTTP/1.1" 200 242103
* F:\\dled-gallery-dl\twitter\twitter_1720534102934229085_1.jpg
ghbook commented 10 months ago

after bit of research, i found

replacing ssl_context.options |= ssl_options with ssl_context.options = ssl.PROTOCOL_TLS_CLIENT fixed my problem. But i have no idea if this is correct. found here: https://github.com/eclipse/paho.mqtt.python/issues/653

https://github.com/mikf/gallery-dl/blob/caf31e751c1caee6e0cb18f8d0c2c043d96fc26c/gallery_dl/extractor/common.py#L778

mikf commented 10 months ago

NOTE: this SSL error occurs only for twitter, remaining extractors working fine.

This should also happen for Patreon and any other site that sets browser = "firefox" by default.

Using ssl.OP_NO_TLSv1 etc is deprecated and should be replaced with minimum_version/maximum_version, which hasn't happened in gallery-dl's code yet.

Anon103 commented 10 months ago

Hello. So what should we do? I've had this error downloading one gallery a few days ago. Should I edit common.py like ghbook commented, or should I wait for a new gallery-dl release that will fix this problem? Thanks a lot!

mikf commented 10 months ago

It's just a warning. Disable it with PYTHONWARNINGS=ignore if it bothers you that much.

Anon103 commented 10 months ago

Oh, I see. My bad. No, it doesn't bother me, I thought it wasn't getting the full res pic. Thanks for the info!