mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
10.84k stars 889 forks source link

[kemonoparty] Heads up #1764

Closed ghost closed 2 years ago

ghost commented 2 years ago

There are new cache servers for files, and the domains have changed. Currently, attempting a download with gallery-dl will result in a 502. You could simply change the domain in the extractor to data1.kemono.party or data2.kemono.party, but considering these may also change in the future, it may be better to hit the main domain (kemono.party) instead and follow the redirects, which will automatically send you to the current data server(s) even if they change.

mikf commented 2 years ago

Thanks for the heads up! Server domain got updated in commit https://github.com/mikf/gallery-dl/commit/c34dbc86bb433d47332c142adab7cbdcd65eb8dc.

The new servers seem to be a bit unstable at the moment, though I'm sure that's going to subside in due time. (429 Too Many Requests, partial responses, timeouts)

deadpixel134 commented 2 years ago

I used this command before latest commit - gallery-dl "https://kemono.party/fantia/user/26637?o=0" --cookies F:\Kemono\cookies.txt

After latest commit when I use same command, It says: [downloader.http][warning] HTTPSConnectionPool(host='data2.kemono.party', port=443): Read timed out. (read timeout=30) (1/5)

So I changed my command as:

Now it says: [downloader.http][warning] '504 Connection Timed Out' for 'https://kemono.party/attachments/fantia/26637/316830/BBwiggle_exHD.mp4' (1/5)

then maybe they have https://data.kemono.party and https://data1.kemono.party, https://data2.kemono.party?

sirlington commented 2 years ago

how do i use and change the new server domain and the commit?

RJFAC commented 2 years ago

According to the admin of kemono on telegram

Shino (shinonome) data.kemono.party is gone, forever data1 and data2 are the file domains now

BTW, I'm getting the same "read timed out" error too.

lodx-xd commented 2 years ago

According to the admin of kemono on telegram

Shino (shinonome) data.kemono.party is gone, forever data1 and data2 are the file domains now

BTW, I'm getting the same "read timed out" error too.

Update your extractor that should work https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/kemonoparty.py Off topic, someone know what is the correct syntax to add more than one exclusion using --filter "extension != " I try --filter "extension != 'clip', 'psd'" and --filter "extension != ('clip', 'psd')" but none of them worked .

Edit: Nvm, looks that --filter "extension not in ('clip', 'psd')" seems to be the correct syntax to use when is more than one exclusion.

sirlington commented 2 years ago

According to the admin of kemono on telegram

Shino (shinonome) data.kemono.party is gone, forever data1 and data2 are the file domains now

BTW, I'm getting the same "read timed out" error too.

Update your extractor that should work https://github.com/mikf/gallery-dl/blob/master/gallery_dl/extractor/kemonoparty.py Off topic, someone know what is the correct syntax to add more than one exclusion using --filter "extension != " I try --filter "extension != 'clip', 'psd'" and --filter "extension != ('clip', 'psd')" but none of them worked .

Edit: Nvm, looks that --filter "extension not in ('clip', 'psd')" seems to be the correct syntax to use when is more than one exclusion.

how to update my textractor?

mikf commented 2 years ago

Version 1.18.3 with the changes from https://github.com/mikf/gallery-dl/commit/c34dbc86bb433d47332c142adab7cbdcd65eb8dc is out.

When you want to use the code from a commit not yet included in an official release, you can always install from the latest commit with the instructions at https://github.com/mikf/gallery-dl#pip or use the executables from https://github.com/mikf/gallery-dl/actions/workflows/executables.yml

MRiCEQB commented 2 years ago

Version 1.18.3 with the changes from c34dbc8 is out.

When you want to use the code from a commit not yet included in an official release, you can always install from the latest commit with the instructions at https://github.com/mikf/gallery-dl#pip or use the executables from https://github.com/mikf/gallery-dl/actions/workflows/executables.yml

No dice for me - using the latest executable from the workflow, but I'm still getting 502 :(

mikf commented 2 years ago

@MRiCEQB are you sure you are using the latest version? is the output of gallery-dl --version "1.18.3" or "1.18.4-dev"? You can also check the file URLs with -g. They should start with https://kemono.party and not https://data.kemono.party.

MRiCEQB commented 2 years ago

@MRiCEQB are you sure you are using the latest version? is the output of gallery-dl --version "1.18.3" or "1.18.4-dev"? You can also check the file URLs with -g. They should start with https://kemono.party and not https://data.kemono.party.

Funny, I just finished writing the following text:

"Yes it says "1.18.4-dev" and my file links are starting with "https://kemono.party/attachments/....." My cookie data is also still the same since the last time it worked :("

I did a final check before posting the comment - and to my own surprise it just worked, I didn't change anything since I posted my last comment. I tried 1.18.3 and 1.18.4-dev yesterday and always got 502 errors upon downloading (I'm using a batch script which always does the same thing).

But for some reason it worked just fine now - I don't know why :D

Thanks for the fix!

lordboosto commented 2 years ago

How would I update my config file to account for the new data1 and data2 domains? image this is what I've got in my config file atm which seemed to be all that was needed from what I saw of prior issues, and it did download some images, but on others it just kept 403ing image I've also checked and I'm on 1.18.3 and my cookies are exactly the same.

mikf commented 2 years ago

@lordboosto you shouldn't need to do anything and your config file looks right, but it seems that gallery-dl can't find and read your config since it's warning about missing DDoS-GUARD cookies.