search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.32k stars 9.95k forks source link

[crunchyroll] Unable to download #25437

Closed narutojawad closed 4 years ago

narutojawad commented 4 years ago

[crunchyroll] Downloading login page ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

it seems like this is cause by a Cloud flare Captcha.

LouisH-760 commented 4 years ago

i'm having this error too, here is the complete log with the verbose flag:

youtube-dl --verbose https://www.crunchyroll.com/the-rising-of-the-shield-hero/episode-1-the-shield-hero-781148
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', 'https://www.crunchyroll.com/the-rising-of-the-shield-hero/episode-1-the-shield-hero-781148']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.05.08
[debug] Python version 3.6.9 (CPython) - Linux-5.5.13-050513-generic-x86_64-with-LinuxMint-19.3-tricia
[debug] exe versions: ffmpeg 4.2.2, ffprobe 4.2.2, phantomjs 5, rtmpdump 2.4
[debug] Proxy map: {}
[crunchyroll] 781148: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/home/louis/.local/lib/python3.6/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/louis/.local/lib/python3.6/site-packages/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 564, in error
    result = self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 756, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
grumpey commented 4 years ago

Same issue.

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--verbose', 'https://www.crunchyroll.com/tower-of-god/episode-8-khuns-strategy-794526'] [debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8 [debug] youtube-dl version 2020.05.29 [debug] Python version 3.8.3 (CPython) - Linux-5.6.14-300.fc32.x86_64-x86_64-with-glibc2.2.5 [debug] exe versions: ffmpeg 4.2.3, ffprobe 4.2.3 [debug] Proxy map: {} [crunchyroll] 794526: Downloading webpage ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/home/grumpey/.local/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage return self._downloader.urlopen(url_or_request) File "/home/grumpey/.local/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2238, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/lib64/python3.8/urllib/request.py", line 531, in open response = meth(req, response) File "/usr/lib64/python3.8/urllib/request.py", line 640, in http_response response = self.parent.error( File "/usr/lib64/python3.8/urllib/request.py", line 563, in error result = self._call_chain(args) File "/usr/lib64/python3.8/urllib/request.py", line 502, in _call_chain result = func(args) File "/usr/lib64/python3.8/urllib/request.py", line 755, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "/usr/lib64/python3.8/urllib/request.py", line 531, in open response = meth(req, response) File "/usr/lib64/python3.8/urllib/request.py", line 640, in http_response response = self.parent.error( File "/usr/lib64/python3.8/urllib/request.py", line 569, in error return self._call_chain(args) File "/usr/lib64/python3.8/urllib/request.py", line 502, in _call_chain result = func(args) File "/usr/lib64/python3.8/urllib/request.py", line 649, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp)

ghost commented 4 years ago

No problems with cookie.

img img2

majorcyto commented 4 years ago

I am trying it with a cookie file, and I am still getting the 403 forbidden page on the login step.

Edit - If I remove my login and ONLY use the cookie file I can access the files. It is only if you include the login step do you get the 403 error now.

ghost commented 4 years ago

Actually the trouble is that they put CloudFlare JsChallenge back on the login page, so youtube-dl is considered by Crunchyroll as a bot.

skid9000 commented 4 years ago

Yay, now we need to find a stupid workaround to bypass this stupid JsChallenge... -_- Thanks again CloudFlare !

skid9000 commented 4 years ago

@remitamine Uh this isn't a duplicate.

Sophira commented 4 years ago

@remitamine That issue which skid9000 commented in was for a completely different project, https://github.com/skid9000/Crunchyroll-Downloader . An issue there isn't a duplicate for this issue. Or am I missing something?

Sophira commented 4 years ago

With regard to the cookie file, I have no luck at all, even if I use a cookie file and don't log in:

$ youtube-dl --version
2020.05.29
$ youtube-dl --cookies cookies.txt https://www.crunchyroll.com/en-gb/my-next-life-as-a-villainess-all-routes-lead-to-doom/episode-4-i-enrolled-in-the-magic-academy-794622
[crunchyroll] 794622: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
narutojawad commented 4 years ago

I tried with a cookies file and it works for me just fine, @Sophira maybe make sure that you are logged in before exporting the cookies file?

avillacis commented 4 years ago

The issue is, before whatever change made by CrunchyRoll, youtube-dl could download a video given an URL to its container webpage, without any need to login, and without manually supplying any cookies file. Now it cannot. There is something about the HTTP request made by youtube-dl that CrunchyRoll does not like, that is different from the one made by the web browser (Firefox in my case). In the case of Firefox, a direct URL to a video will cause www.crunchyroll.com to first respond with a 302 Found, some cookies set, and a redirect to the same URL which then loads normally. In the case of youtube-dl, it instead responds with a 301 Moved Permanently, and when the redirect is followed, it responds with 403 Forbidden.

Note that Firefox can show the CrunchyRoll webpage and can also start playback, without any need to login into any session. Therefore, technically a valid login should not be required for youtube-dl to download the data.

I have tried installing a Firefox extension that exports the cookies from an anonymous (not logged-in) CrunchyRoll page view into a cookies.txt file, and loading this from youtube-dl, and it does not work - it still responds with 301 and then 403 Forbidden.

I have noticed that Firefox uses HTTP/2 for its CrunchyRoll requests, and youtube-dl uses HTTP/1.1 . Could be one difference worth investigating.

ArturiaPendragon commented 4 years ago

It's working with cookies here the details : #25294

ArturiaPendragon commented 4 years ago

@skid9000 Yes i know, i have this two problems Captcha + Unicode problems with youtube-dl. I fixed that last night with pip3 install (and python3) + cookies. (I changed the user-agent as well).

skid9000 commented 4 years ago

That's still a problem, you need a cookie file to download anything. Which means we need to find a solution to solve this CloudFlare check in Python.

ArturiaPendragon commented 4 years ago

Crunchyroll add hCaptcha ?

Maybe this helps : https://blog.skk.moe/post/bypass-hcaptcha/#Cloudflare-Captcha Or this : https://github.com/VeNoMouS/cloudscraper

skid9000 commented 4 years ago

I think cloudscraper was used in a pr a long time ago but got refused because it's not unlicensed. (don't quote me on that tho)

avillacis commented 4 years ago

Update: the youtube-dl HTTPS request is (rightfully) triggering the Cloudflare anti-bot protection by keying on the SSL use of TLS 1.3 by modern browsers vs. TLS 1.2 or older by youtube-dl. That is, Cloudflare probably thinks that the User-Agent advertised by youtube-dl must be used alongside TLS 1.3 or later, and flags the request as a bot if this does not happen.

Evidence: given the following text file (in DOS line-ending format for HTTP protocol specification):

GET / HTTP/1.1 Host: www.crunchyroll.com User-Agent: Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:76.0) Gecko/20100101 Firefox/76.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/;q=0.8 Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3 Accept-Encoding: gzip, deflate, br DNT: 1 Connection: keep-alive Upgrade-Insecure-Requests: 1

The following command triggers the Cloudflare blocking: (cat x0.txt && sleep 5) | openssl s_client www.crunchyroll.com:443 ... but this one does not: (cat x0.txt && sleep 5) | openssl s_client -tls1_3 www.crunchyroll.com:443

Note: openssl s_client uses TLS 1.2 by default. Also, Firefox fills the ALPN field (settable via "-alpn http/1.1" in the openssl client), but Cloudflare is not (yet) keying on that.

TLDR: the CrunchyRoll extractor must find some way to force TLS 1.3 SSL negotiation when requesting the pages.

Parent5446 commented 4 years ago

Is this issue supposed to still be closed?

narutojawad commented 4 years ago

Is this issue supposed to still be closed? No, and it's not duplicate.

Parent5446 commented 4 years ago

Also, FWIW

TLDR: the CrunchyRoll extractor must find some way to force TLS 1.3 SSL negotiation when requesting the pages.

As a quick confirmation, if I edit the make_HTTPS_handler function in utils.py, and add the following line when generating the TLS context:

context.options |= ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1_2

It makes it past the login page finally, but then gives me the following output:

[debug] System config: []
[debug] User config: ['--cookies', '/home/parent5446/cookies.txt', '--retries', 'infinite']
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.crunchyroll.com/sing-yesterday-for-me/episode-1-a-misfits-attempt-at-self-reform-794606', '--cookies', '/home/parent5446/cookies.txt']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.05.29
[debug] Git HEAD: bef4688c7
[debug] Python version 3.8.3 (CPython) - Linux-5.6.14-300.fc32.x86_64-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.2.3, ffprobe 4.2.3
[debug] Proxy map: {}
[crunchyroll] 794606: Downloading webpage
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-frFR information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-esES information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-arME information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-esLA information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-enUS information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-ruRU information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-ptBR information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-itIT information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading adaptive_hls-audio-jaJP-hardsub-deDE information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading media info for 360p
[crunchyroll] 794606: Downloading stream info for 360p
[crunchyroll] 794606: Downloading m3u8 information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading media info for 480p
[crunchyroll] 794606: Downloading stream info for 480p
[crunchyroll] 794606: Downloading m3u8 information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading media info for 720p
[crunchyroll] 794606: Downloading stream info for 720p
[crunchyroll] 794606: Downloading m3u8 information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
[crunchyroll] 794606: Downloading media info for 1080p
[crunchyroll] 794606: Downloading stream info for 1080p
[crunchyroll] 794606: Downloading m3u8 information
WARNING: Failed to download m3u8 information: <urlopen error TLS/SSL connection has been closed (EOF) (_ssl.c:1108)>
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/home/parent5446/Repositories/youtube-dl/youtube_dl/YoutubeDL.py", line 797, in extract_info
    ie_result = ie.extract(url)
  File "/home/parent5446/Repositories/youtube-dl/youtube_dl/extractor/common.py", line 530, in extract
    ie_result = self._real_extract(url)
  File "/home/parent5446/Repositories/youtube-dl/youtube_dl/extractor/crunchyroll.py", line 574, in _real_extract
    self._sort_formats(formats, ('preference', 'language_preference', 'height', 'width', 'tbr', 'fps'))
  File "/home/parent5446/Repositories/youtube-dl/youtube_dl/extractor/common.py", line 1350, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

My guess is that, for whatever stupid reason, TLS 1.3 is necessary for the extractor to bypass Cloudflare, but then it completely breaks when accessing the video itself. :cry:

Parent5446 commented 4 years ago

(I'm an idiot, much simpler to just context.minimum_version = ssl.TLSVersion.TLSv1_3)

Parent5446 commented 4 years ago

Update: I tried getting it to work by adding two different HTTPS handlers into the URL chain, one supporting only TLS 1.3, and another with the default settings (in that order), and it still fails with the same error messages.

Sophira commented 4 years ago

The context settings appear to be retained through different calls. I was able to get it to work (in a very hacky, proof-of-concept kind of way) by forcing TLS v1.3 for www.crunchyroll.com and allowing TLS v1.2 for everything else.

To be specific, I put this code into the YoutubeDLHTTPSHandler.https_open function before the return:

        if req.host == 'www.crunchyroll.com':
            kwargs['context'].options |= ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1 | ssl.OP_NO_TLSv1_2
        else:
            kwargs['context'].options |= ssl.OP_NO_SSLv2 | ssl.OP_NO_SSLv3 | ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1
            kwargs['context'].options &= ~ssl.OP_NO_TLSv1_2

I would not recommend this at all for production because it would interfere with other handlers, but as a proof-of-concept it demonstrates it working.

skid9000 commented 4 years ago

Can someone re-open this issue and remove the "duplicate" tag ? @dstftw @remitamine

avillacis commented 4 years ago

It appears that the Cloudflare change that denied access to unpatched youtube-dl has been reverted. Please somebody else confirm.

skid9000 commented 4 years ago

Still 403 here.

ghost commented 4 years ago

When I get out of drinks, I'll tell you if it works.

avillacis commented 4 years ago

I repeated the openssl s_client test. It appears that now the HTTPS connection is negotiated at TLS 1.3 level, at least for me, without the need to force the TLS version through the command line. However, I can also force TLS 1.2 with the "-tls1_2" switch, and the request is now accepted again. So I am now confused.

@skid9000 Do you have access to the "openssl" program? If you use any flavor of Linux, you probably do. Are you able to repeat the openssl test on your side?

Sophira commented 4 years ago

Still 403ing for me with stock youtube-dl from git.

Sophira commented 4 years ago

@avillacis What are the contents of your x0.txt file? I want to make sure I repeat exactly the same test you did.

avillacis commented 4 years ago

Update: the youtube-dl HTTPS request is (rightfully) triggering the Cloudflare anti-bot protection by keying on the SSL use of TLS 1.3 by modern browsers vs. TLS 1.2 or older by youtube-dl. That is, Cloudflare probably thinks that the User-Agent advertised by youtube-dl must be used alongside TLS 1.3 or later, and flags the request as a bot if this does not happen.

Evidence: given the following text file (in DOS line-ending format for HTTP protocol specification):

GET / HTTP/1.1 Host: www.crunchyroll.com User-Agent: Mozilla/5.0 (X11; Fedora; Linux x8664; rv:76.0) Gecko/20100101 Firefox/76.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,/_;q=0.8 Accept-Language: es-ES,es;q=0.8,en-US;q=0.5,en;q=0.3 Accept-Encoding: gzip, deflate, br DNT: 1 Connection: keep-alive Upgrade-Insecure-Requests: 1

The following command triggers the Cloudflare blocking: (cat x0.txt && sleep 5) | openssl s_client www.crunchyroll.com:443 ... but this one does not: (cat x0.txt && sleep 5) | openssl s_client -tls1_3 www.crunchyroll.com:443

Note: openssl s_client uses TLS 1.2 by default. Also, Firefox fills the ALPN field (settable via "-alpn http/1.1" in the openssl client), but Cloudflare is not (yet) keying on that.

TLDR: the CrunchyRoll extractor must find some way to force TLS 1.3 SSL negotiation when requesting the pages.

The contents were quoted here.

skid9000 commented 4 years ago

I can't get an HTTP error code with your openssl commands 🤔

Sophira commented 4 years ago

Ah, thanks, I'm sorry I missed that.

Without the -tls1_3 flag, I get a 403 error. With the flag, I get a 302 to /es-es.

werewolf004 commented 4 years ago

So no news for this issue?

deathzombee commented 4 years ago

with a pull pending "Detect the cf challenge and try solve it if cfscrape is available." https://github.com/ytdl-org/youtube-dl/pull/20528

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--verbose', 'https://www.crunchyroll.com/skull-face-bookseller-honda-san/episode-4-mission-an-outside-work-ish-job-777825'] [debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252 [debug] youtube-dl version 2020.01.24 [debug] Python version 3.8.0 (CPython) - Windows-10-10.0.18362-SP0 [debug] exe versions: ffmpeg git-2019-12-02-968c4cb, ffprobe git-2019-12-02-968c4cb [debug] Proxy map: {} [crunchyroll] 777825: Downloading webpage ERROR: Unable to extract video_title; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last): File "c:\users\dz\appdata\local\programs\python\python38\lib\site-packages\youtube_dl-2020.1.24-py3.8.egg\youtube_dl\YoutubeDL.py", line 796, in extract_info ie_result = ie.extract(url) File "c:\users\dz\appdata\local\programs\python\python38\lib\site-packages\youtube_dl-2020.1.24-py3.8.egg\youtube_dl\extractor\common.py", line 531, in extract ie_result = self._real_extract(url) File "c:\users\dz\appdata\local\programs\python\python38\lib\site-packages\youtube_dl-2020.1.24-py3.8.egg\youtube_dl\extractor\crunchyroll.py", line 444, in _real_extract video_title = self._html_search_regex( File "c:\users\dz\appdata\local\programs\python\python38\lib\site-packages\youtube_dl-2020.1.24-py3.8.egg\youtube_dl\extractor\common.py", line 1040, in _html_search_regex res = self._search_regex(pattern, string, name, default, fatal, flags, group) File "c:\users\dz\appdata\local\programs\python\python38\lib\site-packages\youtube_dl-2020.1.24-py3.8.egg\youtube_dl\extractor\common.py", line 1031, in _search_regex raise RegexNotFoundError('Unable to extract %s' % _name) youtube_dl.utils.RegexNotFoundError: Unable to extract video_title; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

RobertusIT commented 4 years ago

403 with crouncyroll, how can i download from it?

can you help me?

glop102 commented 4 years ago

@RobertusIT Update python to 3.7+ This is a problem with TLS and cloudflare expecting version 1.3 for the useragent. Python 3.7 adds proper support for TLS1.3

skid9000 commented 4 years ago 
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--username', 'PRIVATE', '--password', 'PRIVATE', '--write-sub', '--sub-lang', 'frFR', '--sub-format', 'ass', '-f', 'best[height=1080]', '--user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/76.0', 'https://www.crunchyroll.com/fr/nyanko-days/episode-12-with-my-cats-728677']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 3.8.5 (CPython) - Linux-4.4.0-18362-Microsoft-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 4.3.1, ffprobe 4.3.1, rtmpdump 2.4
[debug] Proxy map: {}
[crunchyroll] Downloading login page
ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/lib/python3.8/site-packages/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/lib/python3.8/site-packages/youtube_dl/YoutubeDL.py", line 2238, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.8/urllib/request.py", line 531, in open
    response = meth(req, response)
  File "/usr/lib/python3.8/urllib/request.py", line 640, in http_response
    response = self.parent.error(
  File "/usr/lib/python3.8/urllib/request.py", line 569, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 502, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.8/urllib/request.py", line 649, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
glop102 commented 4 years ago 
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.crunchyroll.com/fr/nyanko-days/episode-12-with-my-cats-728677', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2020.07.28
[debug] Python version 3.7.8 (CPython) - Linux-5.7.4-gentoo-x86_64-AMD_Ryzen_7_2700X_Eight-Core_Processor-with-gentoo-2.7
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4
[debug] Proxy map: {}
[crunchyroll] 728677: Downloading webpage
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-ptBR information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-esES information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-itIT information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-frFR information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-arME information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-deDE information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-enUS information
[crunchyroll] 728677: Downloading adaptive_hls-audio-jaJP-hardsub-esLA information
[crunchyroll] 728677: Downloading media info

Example of it working for me.

After trying a few more things, I believe the problem is the attempt to use an account.

This works

[debug] Command-line args: ['https://www.crunchyroll.com/fr/nyanko-days/episode-12-with-my-cats-728677', '--verbose', '--user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/76.0', '--sub-lang', 'frFR', '-f', 'best[height=1080]']

This does not work

[debug] Command-line args: ['https://www.crunchyroll.com/fr/nyanko-days/episode-12-with-my-cats-728677', '--verbose', '--user-agent', 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/76.0', '--sub-lang', 'frFR', '-f', 'best[height=1080]', '--username', 'PRIVATE', '--password', 'PRIVATE']

More minimal examples Working:

[debug] Command-line args: ['https://www.crunchyroll.com/fr/nyanko-days/episode-12-with-my-cats-728677', '--verbose']

403:

[debug] Command-line args: ['https://www.crunchyroll.com/fr/nyanko-days/episode-12-with-my-cats-728677', '--verbose', '--username', 'PRIVATE', '--password', 'PRIVATE']

In short - do not have --username and --password arguments when trying to use crunchyroll. I do not know if using .netrc file is any better but it is probable that it has the exact same problem. You may want to test with cookies after you logged in with a browser.

omegahack0 commented 3 years ago

I dont know if you guys are experiencing this, but it seems to work when I try to download an individual episode as opposed to trying to download the entire playlist (series) so for example youtube-dl https://www.crunchyroll.com/jujutsu-kaisen/episode-1-ryomen-sukuna-797865 - Works for me youtube-dl https://www.crunchyroll.com/jujutsu-kaisen - returns the 403

ErikUden commented 2 years ago

@majorcyto

Edit - If I remove my login and ONLY use the cookie file I can access the files. It is only if you include the login step do you get the 403 error now.

This also worked for me, thanks!