mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.69k stars 954 forks source link

[site support request] furry.booru.org support #1477

Closed VanTuz5 closed 1 month ago

VanTuz5 commented 3 years ago

Please add support for furry.booru.org. Yes, this is actually a mirror of the e621, but there are arts that have been removed from there.

Twi-Hard commented 3 years ago

There's thousands of boorus on booru.org and I think they all use the same api. If this gets supported it should probably support any of them. booru.org boorus can also have custom domain names. https://booru.org/top

VanTuz5 commented 3 years ago

Not always. There is Gelbooru, there is Danbooru, there is Philomena - and these are all different engines that need a different approach.

kattjevfel commented 3 years ago

595bdaa4bee5c9dd0b5e5849054a87a08f345c3d should have addressed this, but I can't figure out how to actually make it work. Especially since furry.booru.org uses danbooru.

The above commit closed all previous "support *.booru.org" issues :p

mikf commented 3 years ago

It is "Running Gelbooru 0.2", so you'd have to use

{
    "extractor": {
        "gelbooru_v02": {
            "furrybooru": {"root": "https://furry.booru.org"}
        }
    }
}

And then there's the issue of it being "protected" by Cloudflare, so you have to jump through some more hoops to be able to access it with gallery-dl. Basically https://github.com/mikf/gallery-dl/issues/1355#issuecomment-791019459, but replace any mention of furaffinity with furry.booru.

VanTuz5 commented 3 years ago

Does this need to be entered into config.json?

mikf commented 3 years ago

Yep. Add the gelbooru_v02 block from above to the extractor section of your config.

VanTuz5 commented 3 years ago

Excuse me, but what exactly should be written there? I just didn't create this file, I wrote the commands manually. Аnd if it is this, then I get this
image

mikf commented 3 years ago

URLs from there contain ampersands &, so you need to put them in quotes. (gallery-dl "https://furry.booru.org/index.php?page=post&s=view&id=12345")

edit: or you backslash-escape them \&, but that's even more tedious.

VanTuz5 commented 3 years ago

image

mikf commented 3 years ago

cmd.exe doesn't like single quotes (and is generally just terrible to work with). You have to use double quotes around the user-agent setting, or you just put it into your config as well.

VanTuz5 commented 3 years ago

Like this? { "extractor": { "gelbooru_v02": { "furrybooru": {"root": "https://furry.booru.org"} "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36 OPR/75.0.3969.171 (Edition Yx 05)" } } }

mikf commented 3 years ago

That'd work, but you have to separate "furrybooru" and "user-agent" with a comma:

{
    "extractor": {
        "gelbooru_v02": {
            "furrybooru": {"root": "https://furry.booru.org"},
            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.114 Safari/537.36 OPR/75.0.3969.171 (Edition Yx 05)"
        }
    }
}
VanTuz5 commented 3 years ago

image

mikf commented 3 years ago

Your user-agent string isn't correct. Go to https://httpbin.org/user-agent and copy it from there. You do have a cf_clearance entry in your cookies file, don't you?

VanTuz5 commented 3 years ago

Yes. And I changed the user-agent just in case, but the result is the same.

catlover1019 commented 2 years ago

it'd still be really nice to have prper support for this site. there's a lot of satuff hard to find elsewhere. @mikf

Hrxn commented 2 years ago

The site in question (furry.booru.org) is a Gelbooru 0.2 instance, so this should already work.

I realize that the example config file used for reference is missing such an example, guess I'll try to whip up a PR real quick. Anyway, this issue can probably be closed nonetheless.

Joebugg commented 2 months ago

It should be noted that the furry.booru.org site just redirects to tbib.org (The Big Image Booru), but this issue still applies to other Boorus with that same code.

Furry Booru is dead.