[nhentai][error] HttpError: '503 Service Temporarily Unavailable'

[onamaetakendesu]

I have used gallery-dl in the past without any problems, but today when I tried to download something, I kept getting this error, and I can't understand why. gallery-dl has been the most fantastic experience ever, and it would be amazing to get it back.

The problem occurs only with nhentai. The error is the same as in the title: [nhentai][error] HttpError: '503 Service Temporarily Unavailable' for 'https://nhentai.net/api/gallery/theholynumbershere'

[mikf]

Nhentai has employed some Cloudflare protection to stop anything that isn't a web browser from accessing its site.

You can circumvent this by grabbing the cf_clearance cookie for nhentai from your browser and using it and your browser's user agent with gallery-dl.

Take a look at these other issues if you need more details: https://github.com/mikf/gallery-dl/issues/2537, https://github.com/mikf/gallery-dl/issues/2633, https://github.com/mikf/gallery-dl/issues/2636

[onamaetakendesu]

Thanks, will it be a one-time thing, or should I add the cookie every single time i use gallery-dl?

Edit: I can't figure out how to make it work. Can you dumb it down for me, please? I am a noob.

[mikf]

You have to specify the cookie every time you run gallery-dl, but you can put it and the user agent string in a config file so it gets loaded automatically.

{
    "extractor": {
        "nhentai": {
            "user-agent": "Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Firefox/102.0",
            "cookies": {"cf_clearance": "XmvlWJHHe2Hb3aCm2HBcQ3JUXh9gL4cW3F5nUqNeuLU-1661872323-0-150"}
        }
    }
}

You obviously have to replace those two values with your own, and you might have to update the cf_clearance value every few hours/days.

You can visit https://httpbin.org/user-agent to get your user agent string. For cookies, take a look at https://github.com/mikf/gallery-dl#cookies if you need help.

[onamaetakendesu]

I looked into the references and replaced the values you mentioned to replace by my own, and saved it to the required path: %APPDATA%\gallery-dl\config.json %USERPROFILE%\gallery-dl\config.json

{ "extractor": { "nhentai": { "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/104.0.0.0 Safari/537.36", "cookies": {"cf_clearance": "nnt97D7vfTUaFwCHoUoYnRTF2UC4Y9r9t42WNqiO30A-1661889331-0-150"} } } }

Idk what I did wrong, but it's still not working. same error. i would like to highlight the way I created the .json was writing it into the notepad and then saved it as .json. maybe that was not the right way. that's all I did.

I really appreciate the time you are taking to help me and hopefully I can understand or make it work properly.

[mikf]

writing it into the notepad and then saved it as .json

My guess is that notepad saved it as .json.txt and hid the .txt part. Try adding some invalid JSON to your config file, like a , as the very first character. If gallery-dl does not complain ([config][warning] Could not parse …), it could not find your config file.

[onamaetakendesu]

I tried as you suggested, and it's detecting the config. Also, from what I noticed, the .json file was saved correctly, considering when I look at it, it says: JSON File

[config][warning] Could not parse 'C:\Users\MyName\AppData\Roaming\gallery-dl\config.json': Expecting value: line 1 column 1 (char 0)

I tried to refresh my cf_clearance but in vain.

[Hrxn]

Seems to work for me...

(latest dev version pulled from repo as of right now..)

Randomly picked one entry from the nhentai front page (upper left corner at the time of my attempt)

PS D:\Temp> gallery-dl -v
Debug  :  gallery-dl -> Version 1.23.1-dev
Debug  :  gallery-dl -> Python 3.10.6 - Windows-10-10.0.19043-SP0
Debug  :  gallery-dl -> requests 2.28.1 - urllib3 1.26.12
usage: gallery-dl [OPTION]... URL...
gallery-dl: error: The following arguments are required: URL
Use 'gallery-dl --help' to get a list of all options.
PS D:\Temp> gallery-dl 'https://nhentai.net/g/417841/'
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_001.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_002.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_003.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_004.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_005.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_006.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_007.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_008.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_009.jpg
* E:\Down\Nhentai\+Galleries\[Kuon].JK.Anal_.CH.01-05.[English].[Castle.TL]_by_['kuon']_417841\417841_010.jpg
                                               ------ SNIP ------

Relevant config parts:

            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36",
            "cookies": {"cf_clearance": "  << cf_clearance cookie taken from browser dev tools here >> "},

I've been using Chrome, obviously, but this should not matter..

[onamaetakendesu]

how? this sucks, why is mine not working? :(

the version I am using is Windows Standalone Executable

[mikf]

Your gallery-dl version does not matter here and your config setup is all how it should be.

It is probably your cf_clearance cookie value that got somehow invalidated. Try getting a new one in a private browser session (Ctrl+Shift+N, go to nhentai, grab your cookie). Alternatively, you could try --cookies-from-browser chrome to import your current browser cookies (might take a while)

I'm still able to download from nhentai with the settings and cookies from https://github.com/mikf/gallery-dl/issues/2878#issuecomment-1231814163. Have you tried using those? I definitely have a different public IP address than I did 2 days ago and it still works, but it might be bound by region.

[Hrxn]

Connectivity issue? Working in browser without any hiccups?

[onamaetakendesu]

WOAH, what kind of sorcery is this!? It's working now; all I did was change/ update the cf_clearance cookie.

Thank you so much to both of you. I downloaded what Hrxn downloaded for the test and was surprised to see it working. I appreciate the help and support from both of you.

Edit: It's back to being dumb again, I don't know why it not working now. Can VPN or anything affect it? I did the test and switched my VPN on to play a game. Then I wanted to try again afterward, after refreshing the cf_clearance cookie, it went back to stopped working.

Edit 2: I have been trying all the options/ methods mentioned above, and it seems that Firefox is the issue for some reason, the crf_clearance from Firefox never works, but for chrome, it works all the time. Weird, and I have no idea at all why it's happening.