mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
10.68k stars 881 forks source link

[Fanbox] Cloudflare challenges #5791

Open biggestsonicfan opened 2 days ago

biggestsonicfan commented 2 days ago

Looks like Fanbox recently updated to add cloudflare challenges? I'm using my cookies from browser with the same browser type and user agent.

EDIT: Only way I've gotten anything to work is by a Japanese VPN

drunk-moe commented 2 days ago

Refresh your cookies and double check your user agent. Just happened to me, got my UA from the requested headers in the network panel and rexported my cookies and it works fine now. A bit annoying but what can you do?

Also - not using any sort of VPN or anything. It worked fine for me once I grabbed my cookies again and used the same User Agent my browser was using.

biggestsonicfan commented 2 days ago

Strangely, it was the "browser": "firefox" key that was the culprit. Removing that resolved my issue.

mikf commented 1 day ago

Removing that resolved my issue.

browser overrides user-agent when present.

bugreporter2334 commented 1 day ago

drunk-moe's method is not quite working for me. spoofing my browser's user agent Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36 and using the same FANBOXSESSID from the cookies still gives me the error [fanbox][warning] Cloudflare challenge [fanbox][error] HttpError: '403 Forbidden' for 'https://api.fanbox.cc/post.listCreator?creatorId=[creator]&limit=10' refreshing the page on my browser does not give a cloudfare challenge.

edit: gallery-dl version for posterity (windows):

$ gallery-dl --version
1.27.1

relevant portions of my config:

    "extractor":
    {
        "base-directory": "./gallery-dl/",
        "parent-directory": false,
        "postprocessors": null,
        "archive": null,
        "cookies": null,
        "cookies-update": true,
        "proxy": null,
        "skip": true,

        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
        "retries": 4,
        "timeout": 30.0,
        "verify": true,
        "fallback": true,

        "sleep": 0,
        "sleep-request": 0,
        "sleep-extractor": 0,

        "path-restrict": "auto",
        "path-replace": "_",
        "path-remove": "\\u0000-\\u001f\\u007f",
        "path-strip": "auto",
        "extension-map": {
            "jpeg": "jpg",
            "jpe" : "jpg",
            "jfif": "jpg",
            "jif" : "jpg",
            "jfi" : "jpg"
        },

        "fanbox":
        {
            "cookies": 
            {
              "FANBOXSESSID": "<sessionid>"
            },
            "postprocessors": 
            [{
              "name": "metadata",
              "event": "post",
              "mode": "json",
              "filename": "{id}.json",
              "directory": "posts"
            }]
        },
ZenoArrows commented 13 hours ago

I've also suddenly started encountering these none of the methods described so far resolve it unfortunately.

biggestsonicfan commented 11 hours ago

"FANBOXSESSID": "<sessionid>"

You are missing both the cf_clearance and __cf_bm cloudflare cookies.

ZenoArrows commented 11 hours ago

You are missing both the cf_clearance and __cf_bm cloudflare cookies.

I've tried that also with the user agent set to be the same as my browser, but it still seems to be able to distinguish between my browser and gallery-dl.

biggestsonicfan commented 9 hours ago

My reply was for @bugreporter2334, I can't see your config file if you don't post it.

bugreporter2334 commented 9 hours ago

"FANBOXSESSID": "<sessionid>"

You are missing both the cf_clearance and __cf_bm cloudflare cookies.

Yep, works now with these two cookies. Here's the syntax:

"fanbox":
        {
            "cookies": 
            {
              "FANBOXSESSID": "fanbox cookie",
              "cf_clearance": "cf cookie1",
              "__cf_bm": "cf cookie 2"
            },
            "postprocessors": 
            [{
              "name": "metadata",
              "event": "post",
              "mode": "json",
              "filename": "{id}.json",
              "directory": "posts"
            }]
        },

On Chrome you can extract them from the same Application tab that you normally get your Fanbox cookies from.