simplymemes / crunchyroll-dl

A fast, modern, and beautiful Crunchyroll downloader.
MIT License
134 stars 22 forks source link

(node:28502) UnhandledPromiseRejectionWarning: Error: Request failed with status code 403 #50

Open ArturiaPendragon opened 4 years ago

ArturiaPendragon commented 4 years ago

Hello here :

crunchyroll-dl -u xxxx -p xxxx -i https://www.crunchyroll.com/sing-yesterday-for-me/episode-3-what-is-love-794608 -l frFR -q 1080p -s soft --vilos -o test.mkv

(node:28502) UnhandledPromiseRejectionWarning: Error: Request failed with status code 403
    at createError (/usr/lib/node_modules/crunchyroll-dl/node_modules/axios/lib/core/createError.js:16:15)
    at settle (/usr/lib/node_modules/crunchyroll-dl/node_modules/axios/lib/core/settle.js:17:12)
    at IncomingMessage.handleStreamEnd (/usr/lib/node_modules/crunchyroll-dl/node_modules/axios/lib/adapters/http.js:236:11)
    at IncomingMessage.emit (events.js:322:22)
    at endReadableNT (_stream_readable.js:1187:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21)
(node:28502) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:28502) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
AlexIsOK commented 4 years ago

In most command line programs, arguments that are more than one character should start with two hyphens rather than one, so you should be able to fix this by changing -vilos to --vilos.

Hope this helps!

Edit: in most programs, if you have -vilos rather than --vilos, it will try to read it like -v -i -l -o -s instead of the intended behavior.

ArturiaPendragon commented 4 years ago

Yes, I made a mistake in the command line, but the error remains the same (Captcha of Cloudflare)

simplymemes commented 4 years ago

I was debugging this for a bit today and couldn't seem to get around it, that's annoying

ArturiaPendragon commented 4 years ago

It's works on youtube-dl with cookie file (no captcha required), but all tools (cr-dl, crunchy-dl, cr-dl, crunchyroll-dl and others) doesn't work with login/password ... (I tried a lot of testing haha..)

Thx to Cloudflare 🥺

AlexIsOK commented 4 years ago

This does not seem to be happening to me (United States). Could this be a regional issue?

Edit: started happening to me on my Windows VM when trying this, but not on my main OS.

AlexIsOK commented 4 years ago

I found out what is most likely causing the issue, the package cloudscraper that this uses has been deprecated. The only other (updated) CF bypass I've seen is written in Python, but I'll look around and see if there's another library to use.

ArturiaPendragon commented 4 years ago

This exist : https://github.com/RyuzakiH/CloudflareSolverRe/pull/29

tomasdiaztoro commented 4 years ago

Im getting this error only when use -s soft

AlexIsOK commented 4 years ago

@ArturiaPendragon that library is made using C#, I'm not sure that you can port that to NodeJS.

I'm looking into libraries for this right now, I'll make a PR if I find anything.

ArturiaPendragon commented 4 years ago

@AlexIsOK He fixed this with change with TLSv1.3 (for login).

https://github.com/anidl/crunchyroll-downloader-nx/commit/8d853d0d8bdcc88b03a2d59ee6fb80fef9c038b9

Maybe this helps?

https://github.com/ytdl-org/youtube-dl/issues/25437

--tls-min-v1.3 set default TLS minimum to TLSv1.3 (default: TLSv1.2)

simplymemes commented 4 years ago

Hmm, not sure if the library this uses supports the manual setting of TLS from first glance, will need more investigation

DavCP commented 4 years ago

This isn't solved yet, right? :(

simplymemes commented 4 years ago

Yeah :( It's getting a lot harder now to work on this since I keep getting captchas from Cloudflare. Hopefully I'll have something but it's take a while

AlexIsOK commented 4 years ago

I'll look into this as well. I got it to (sort of) work by calling py3 from the Node runtime and having it use this Python cloudscraper but that requires the user to have Python3 installed and has to check to see if the PIP3 dependencies are met.

Edit: those changes are available here if you want to see.

Edit 2: I haven't committed the changes that work yet, so I'll do that right now.

AlexIsOK commented 4 years ago

@simplymemes on my fork on branch patch-2, I'm getting an error with this regex. CR may have changed the div class of what you're trying to match. Downloading single episodes works fine for me though.

I was able to make a (horrible) fix by running the Python cloudscraper I mentioned before (which works very well, but the user must have pip3 and python3 installed, and must have pip3 install cloudscraper).

Unless there's a cloudscraper for NodeJS that works, I don't see an easy way to do this, though I could easily be missing something.

PS. the cloudscraper can get the content of the CF page, but the regex doesn't match.

Owyii commented 4 years ago

I might be interested in the python solution, once i have pip3, python3 and pip3 install the only thing needed is running the cloudscraper?

AlexIsOK commented 4 years ago

@Owyii It doesn't work until the regex is fixed, and I found that the downloader works a lot better if you download the individual episode rather than all of them or series at a time. Maybe this could be helpful for a future patch?

Owyii commented 4 years ago

@AlexIsOK I am able to download the single episode if the subtitle are hardsubbed, however as soon i try to download with softsub I get error 403