[Site Request] derpibooru.org

[3k2]

Hey

would be nice to have derpibooru support please :)

[Twi-Hard]

They have an API https://derpibooru.org/pages/api

[3k2]

how's this going?

[marius851000]

I need something like that. I'll try to work on that, but I don't have any experience with gallery-dl (althougt I did some stuff on youtube-dl).

[ghost]

how's this going?

There's a dedicated downloader for that.

[rautamiekka]

how's this going?

There's a dedicated downloader for that.

Not an option since it's Window$-only, and graphical.

[Hrxn]

Is philomena actually any good?

[rautamiekka]

^ Seems that way; I've used https://github.com/mcilya/DBooru to get my likes/favs, which uses the API, and it works perfectly. Don't bother with https://github.com/joshua-stone/DerPyBooru unless you're using it to code your own Derpibooru Client, it's only an API wrapper.

Although, it doesn't support URL's, only keywords and likes/favs, via a very hardcoded TUI.

(Philomena's problem's it's hardcoded for PostgreSQL, the way I understand, but shouldn't be much of a problem to modify for MariaDB.)

[mikf]

https://github.com/mikf/gallery-dl/commit/fa8ee6eac49678ba5decf4dc389fda762f158e10 adds initial support for general searches, galleries, and single images.

By default it uses the site's "Default" filter (should it use the "Everything" filter or similar instead?), but it is possible to set an API key or an explicit filter ID to change that.

Let me know what else would be useful to have, and any other potential improvements like for instance default file names.

[3k2]

fa8ee6e adds initial support for general searches, galleries, and single images.

By default it uses the site's "Default" filter (should it use the "Everything" filter or similar instead?), but it is possible to set an API key or an explicit filter ID to change that.

Let me know what else would be useful to have, and any other potential improvements like for instance default file names.

what am I doing wrong here I can just download SFW content even tho I have set my API key?

I made new line to my gallery-dl.conf like this

        "derpibooru":
        {
            "extractor.derpibooru.filter": "56027",
            "extractor.derpibooru.api-key": "API-key"
        },

I tried with everything filter only and only API-key but no difference.

[mikf]

It's only "api-key" and "filter", inside "derpibooru", inside "extractor", hence extractor.derpibooru.filter in the docs:

        "derpibooru":
        {
            "filter": "56027",
            "api-key": "API-key"
        },

[3k2]

It's only "api-key" and "filter", inside "derpibooru", inside "extractor", hence extractor.derpibooru.filter in the docs:

        "derpibooru":
        {
            "filter": "56027",
            "api-key": "API-key"
        },

hmm I did this but it still seems to be only downloading SFW content not sure what's going on.

 gallery-dl -v --no-download -d /mnt/yiffyraid/Yiffer/Albums/ https://derpibooru.org/search\?q\=faved_by_id%3A454599
[gallery-dl][debug] Version 1.16.3
[gallery-dl][debug] Python 3.9.1 - Linux-5.10.6-zen1-1-zen-x86_64-with-glibc2.32
[gallery-dl][debug] requests 2.25.1 - urllib3 1.26.1
[gallery-dl][debug] Starting DownloadJob for 'https://derpibooru.org/search?q=faved_by_id%3A454599'
[derpibooru][debug] Using DerpibooruSearchExtractor for 'https://derpibooru.org/search?q=faved_by_id%3A454599'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): derpibooru.org:443
[urllib3.connectionpool][debug] https://derpibooru.org:443 "GET /api/v1/json/search/images?q=faved_by_id%3A454599&page=1&per_page=50 HTTP/1.1" 200 None

I tried another gallery with only nsfw content here's the result

➜ gallery-dl -v --no-download -d /mnt/yiffyraid/Yiffer/Albums/ https://derpibooru.org/galleries/11761
[gallery-dl][debug] Version 1.16.3
[gallery-dl][debug] Python 3.9.1 - Linux-5.10.6-zen1-1-zen-x86_64-with-glibc2.32
[gallery-dl][debug] requests 2.25.1 - urllib3 1.26.1
[gallery-dl][debug] Starting DownloadJob for 'https://derpibooru.org/galleries/11761'
[derpibooru][debug] Using DerpibooruGalleryExtractor for 'https://derpibooru.org/galleries/11761'
[urllib3.connectionpool][debug] Starting new HTTPS connection (1): derpibooru.org:443
[urllib3.connectionpool][debug] https://derpibooru.org:443 "GET /api/v1/json/search/galleries?q=id%3A11761 HTTP/1.1" 200 None
[derpibooru][debug] Sleeping for 0.999 seconds
[urllib3.connectionpool][debug] https://derpibooru.org:443 "GET /api/v1/json/search/images?sd=desc&sf=gallery_id%3A11761&q=gallery_id%3A11761&page=1&per_page=50 HTTP/1.1" 200 41

[mikf]

Maybe gallery-dl isn't automatically loading your config file because it's not at the right location?

Try using --config /path/to/your/config.file and see if that helps. You can also set the filter value with -o filter=56027. This would also work for api-key, but pasting that value to the cmdline might be a bit too unwieldy.

Going back to my question from before: Would it be better to use the "Everything" filter or similar by default?

[3k2]

Maybe gallery-dl isn't automatically loading your config file because it's not at the right location?

Try using --config /path/to/your/config.file and see if that helps. You can also set the filter value with -o filter=56027. This would also work for api-key, but pasting that value to the cmdline might be a bit too unwieldy.

Going back to my question from before: Would it be better to use the "Everything" filter or similar by default?

yes it should use by default "Everything"

for some reason it is not reading the config file correclty even tho when I directly do --config command to the file it seems to ignore the filter and api-key values? not sure what's going on.

[mikf]

yes it should use by default "Everything"

https://github.com/mikf/gallery-dl/commit/01b9ccd4de0c91ee572486916531add0fc28ab0f

for some reason it is not reading the config file correclty even tho when I directly do --config command to the file it seems to ignore the filter and api-key values? not sure what's going on.

Hard to tell without seeing your config as a whole, but the following works as a minimal example:

{
    "extractor": {
        "derpibooru": {
            "filter": 12345
        }
    }
}

$ gallery-dl -v --ignore-config -c config.json https://derpibooru.org/tags/TAG
[urllib3.connectionpool][debug] https://derpibooru.org:443 "GET /api/v1/json/search/images?q=TAG&page=1&per_page=50&filter_id=12345 HTTP/1.1" 200 None