[chaturbate] HTTP Error 403: Forbidden for TOR socks5 proxy

marabu88 commented 4 years ago

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2019.11.28
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log

OLD COMPILED EXE (2019.05.11, Python version 3.4.4)

...>CB_youtube-dl.exe -v
[debug] System config: []
[debug] User config: ['--cookies', 'C:\\Users\\...\\AppData\\Roaming\\youtube-dl\\cookies.txt', '--sleep-interval', '1', '--max-sleep-interval', '2']
[debug] Custom config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2019.05.11
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg N-89794-gc51301db14, rtmpdump 2.4
[debug] Proxy map: {}
Usage: CB_youtube-dl.exe [OPTIONS] URL [URL...]

...>CB_youtube-dl.exe --ignore-config --proxy socks5://192.168.1.2:9050 https://chaturbate.com/lucy_lew/
[Chaturbate] lucy_lew: Downloading webpage
ERROR: Room is currently offline

=================================================================

VERSION WITH PYTHON 3.7.4 (and LAST ALSO)

...>C:\Python37\Scripts\youtube-dl.exe -v
[debug] System config: []
[debug] User config: ['--cookies', 'C:\\Users\\...\\AppData\\Roaming\\youtube-dl\\cookies.txt', '--sleep-interval', '1', '--max-sleep-interval', '2']
[debug] Custom config: []
[debug] Command-line args: ['-v']
[debug] Encodings: locale cp1251, fs utf-8, out utf-8, pref cp1251
[debug] youtube-dl version 2019.05.11
[debug] Python version 3.7.4 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg N-89794-gc51301db14, rtmpdump 2.4
[debug] Proxy map: {}
Usage: youtube-dl [OPTIONS] URL [URL...]

...>C:\Python37\Scripts\youtube-dl.exe -v --ignore-config --proxy socks5://192.168.1.2:9050 https://chaturbate.com/lucy_lew/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--ignore-config', '--proxy', 'socks5://192.168.1.2:9050', 'https://chaturbate.com/lucy_lew/']
[debug] Encodings: locale cp1251, fs utf-8, out utf-8, pref cp1251
[debug] youtube-dl version 2019.05.11
[debug] Python version 3.7.4 (CPython) - Windows-10-10.0.18362-SP0
[debug] exe versions: ffmpeg N-89794-gc51301db14, rtmpdump 2.4
[debug] Proxy map: {'http': 'socks5://192.168.1.2:9050', 'https': 'socks5://192.168.1.2:9050'}
[Chaturbate] lucy_lew: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "c:\python37\lib\site-packages\youtube_dl\extractor\common.py", line 626, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "c:\python37\lib\site-packages\youtube_dl\YoutubeDL.py", line 2227, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "c:\python37\lib\urllib\request.py", line 531, in open
    response = meth(req, response)
  File "c:\python37\lib\urllib\request.py", line 641, in http_response
    'http', request, response, code, msg, hdrs)
  File "c:\python37\lib\urllib\request.py", line 569, in error
    return self._call_chain(*args)
  File "c:\python37\lib\urllib\request.py", line 503, in _call_chain
    result = func(*args)
  File "c:\python37\lib\urllib\request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

Description

READ THIS FIRST !!! I checked different options to understand why I get a 403 error for chaturbate if I use TOR as a proxy. even if I select some proxy that does not cause an error, then it appears after a few minutes on ANY proxy. at the same time, if I log in through the TOR browser (with the same proxy), then I don’t have any problems, though I don’t enter captcha and do not go through the authentication process! why then do i see this error in youtube-dl? in the end, I tried to run youtube-dl instead of the compiled version as the original program and got the same error. maybe something changed in python itself? no need to uproot this topic as long as the gap exists. or can you not confirm its presence? check at least before closing this topic !!!

marabu88 commented 4 years ago

The result of my experiments now!

I use commands like this pip3.4 install -I youtube-dl==2019.05.11 and try install old version for python 3.4 and 3.7

Python 3.7 youtube-dl 2019.11.28 - Error 403 Python 3.4 youtube-dl 2019.11.28 - Error 403 Python 3.7 youtube-dl 2019.05.11 - Error 403 Python 3.4 youtube-dl 2019.05.11 - All OK

C:\Python34\Scripts>youtube-dl.exe -v --ignore-config --proxy socks5://192.168.1.2:9050 https://chaturbate.com/lucy_lew/
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--ignore-config', '--proxy', 'socks5://192.168.1.2:9050', 'https://chaturbate.com/lucy_lew/']
[debug] Encodings: locale cp1251, fs mbcs, out cp866, pref cp1251
[debug] youtube-dl version 2019.05.11
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg N-89794-gc51301db14, rtmpdump 2.4
[debug] Proxy map: {'http': 'socks5://192.168.1.2:9050', 'https': 'socks5://192.168.1.2:9050'}
[Chaturbate] lucy_lew: Downloading webpage
ERROR: Room is currently offline
Traceback (most recent call last):
  File "c:\python34\lib\site-packages\youtube_dl\YoutubeDL.py", line 796, in extract_info
    ie_result = ie.extract(url)
  File "c:\python34\lib\site-packages\youtube_dl\extractor\common.py", line 529, in extract
    ie_result = self._real_extract(url)
  File "c:\python34\lib\site-packages\youtube_dl\extractor\chaturbate.py", line 57, in _real_extract
    raise ExtractorError(error, expected=True)
youtube_dl.utils.ExtractorError: Room is currently offline

marabu88 commented 4 years ago

My solution (but I don't like it). created the Frankenstein monster - used Python 3.4, youtube-dl 2019.05.11 and the extractor from youtube-dl 2019.11.28. And in this case, everything works without problems:

C:\Python34\Scripts\youtube-dl.exe --ignore-config -i --proxy socks5://192.168.1.2:9050 --hls-use-mpegts -o juicy_jesss.ts "https://chaturbate.com/juicy_jesss"

[Chaturbate] juicy_jesss: Downloading webpage
[Chaturbate] juicy_jesss: Downloading m3u8 information
[download] Destination: juicy_jesss.ts
WARNING: ffmpeg does not support SOCKS proxies. Downloading is likely to fail. Consider adding --hls-prefer-native to your command.
ffmpeg version N-89794-gc51301db14 Copyright (c) 2000-2018 the FFmpeg developers
  built with gcc 7.2.0 (GCC)
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-bzlib --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-amf --enable-cuda --enable-cuvid --enable-d3d11va --enable-nvenc --enable-dxva2 --enable-avisynth --enable-libmfx
  libavutil      56.  7.100 / 56.  7.100
  libavcodec     58.  9.100 / 58.  9.100
  libavformat    58.  3.100 / 58.  3.100
  libavdevice    58.  0.100 / 58.  0.100
  libavfilter     7. 11.101 /  7. 11.101
  libswscale      5.  0.101 /  5.  0.101
  libswresample   3.  0.101 /  3.  0.101
  libpostproc    55.  0.100 / 55.  0.100
[hls,applehttp @ 000002ddb48e9b40] Opening 'https://edge50.stream.highwebmedia.com/live-hls/amlst:juicy_jesss-sd-cf08a1a2075cdd0263fa1463686b4183e9917d05bca30dc4f9d2eb04c46c9613_trns_h264/media_w584467902_b1128000_t64RlBTOjMwLjA=_544.ts' for reading
Input #0, hls,applehttp, from 'https://edge50.stream.highwebmedia.com/live-hls/amlst:juicy_jesss-sd-cf08a1a2075cdd0263fa1463686b4183e9917d05bca30dc4f9d2eb04c46c9613_trns_h264/chunklist_w584467902_b1128000_t64RlBTOjMwLjA=.m3u8':
  Duration: N/A, start: 814.534000, bitrate: N/A
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 720x540 [SAR 1:1 DAR 4:3], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:2: Audio: aac (LC) ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp
    Metadata:
      variant_bitrate : 0
Output #0, mpegts, to 'file:J:\CB\videos\juicy_jesss_20191128213023.ts.part':
  Metadata:
    encoder         : Lavf58.3.100
    Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 720x540 [SAR 1:1 DAR 4:3], q=2-31, 30 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
    Stream #0:1: Audio: aac (LC) ([15][0][0][0] / 0x000F), 16000 Hz, mono, fltp
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:1 -> #0:0 (copy)
  Stream #0:2 -> #0:1 (copy)
Press [q] to stop, [?] for help
skipping 2 segments ahead, expired from playlists
[https @ 000002ddb48ed600] Opening 'https://edge50.stream.highwebmedia.com/live-hls/amlst:juicy_jesss-sd-cf08a1a2075cdd0263fa1463686b4183e9917d05bca30dc4f9d2eb04c46c9613_trns_h264/media_w584467902_b1128000_t64RlBTOjMwLjA=_547.ts' for reading

I ask for help figuring out why this is happening. Is it possible to find out exactly where the problem arises? I do not know enough about python and this program for this :(

Alion548 commented 4 years ago

I also encountered the same problem

ytdl-org / youtube-dl