search

mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.8k stars 967 forks source link

[Kemono, request] Add post_id field for files #2984

Closed Skyofflad closed 2 years ago

Skyofflad commented 2 years ago

After the recent change I cannot use post id for "archive-format" and "metadata" postprocessor. They use attachment id instead. Can you add "post_id" to "Keywords for filenames", please?

afterdelight commented 2 years ago

when is that recent change?

biggestsonicfan commented 2 years ago

Seems like today. If I was logging specifically I would know exactly because everything 403's now without the post_id. This image is a 403 but if you add ?f=0.png it becomes accessible.

Amie-chan commented 2 years ago

Can confirm, was downloading fine yesterday but getting 403s today

Sherman-Liu commented 2 years ago

@Amie-chan Same problem, already checked cookies, doesn't seem like a DDOSGuard related error.

afterdelight commented 2 years ago

Seems like today. If I was logging specifically I would know exactly because everything 403's now without the post_id. This image is a 403 but if you add ?f=0.png it becomes accessible.

When i try to download the image with idm it became 0.png. Need a proper fix. Btw what's the name of the art author?

biggestsonicfan commented 2 years ago

When i try to download the image with idm it became 0.png. Need a proper fix. Btw what's the name of the art author?

I assume this was an implementation by kemono so users of the site (not gallery-dl) could just save via original name.

Sherman-Liu commented 2 years ago

@afterdelight You could use JDownloader to download in a pinch, just paste the user page link into the JDownloader, it'll automatically index the post name as folder name, and fix filename properly.

图片

mikf commented 2 years ago

New anti-botting measure. Please wait until things are sorted out, then it should work.

According to https://chan.kemono.party/kemono/res/22796.html#24029

@Skyofflad the post_id for kemono has always just been called id, and the last change didn't touch any of that (09a5cc61). It merely added new metadata fields, non were changed or removed.

Skyofflad commented 2 years ago

@Skyofflad the post_id for kemono has always just been called id, and the last change didn't touch any of that (09a5cc6). It merely added new metadata fields, non were changed or removed.

As an example, here post id should be 9387365 but in "keywords for filenames" it is equal to attachments[][id] (995376)

./gallery-dl.bin -K https://kemono.party/patreon/user/402938/post/9387365
Keywords for directory names:
-----------------------------
added
  Wed, 08 Jul 2020 03:04:29 GMT
attachments[][hash]
  c8b963b60373991c8ae6cd49dbfb4a988acaff7f64e5fa0acd7e0d5fbf8867cb
attachments[][id]
  995376
attachments[][name]
  Inque.jpg
attachments[][path]
  /c8/b9/c8b963b60373991c8ae6cd49dbfb4a988acaff7f64e5fa0acd7e0d5fbf8867cb.jpg
attachments[][type]
  attachment
category
  kemonoparty
content
  Check the attached file!  thank you!!!
count
  1
date
  2017-04-27 22:11:25
edited
  Thu, 27 Apr 2017 22:11:25 GMT
file[name]
  Inque_Patreon.jpg
file[path]
  /8d/fb/8dfbc9b141593f71dd89632a09c2f2715af851810ee3ecf21e75909ef715fd35.jpg
id
  9387365
published
  Thu, 27 Apr 2017 22:11:25 GMT
service
  patreon
shared_file
  False
subcategory
  patreon
title
  [High Res File] Inque from Batman Beyond Patreon Commission.
user
  402938
username
  reiq

Keywords for filenames and --filter:
------------------------------------
added
  Wed, 08 Jul 2020 03:04:29 GMT
attachments[][hash]
  c8b963b60373991c8ae6cd49dbfb4a988acaff7f64e5fa0acd7e0d5fbf8867cb
attachments[][id]
  995376
attachments[][name]
  Inque.jpg
attachments[][path]
  /c8/b9/c8b963b60373991c8ae6cd49dbfb4a988acaff7f64e5fa0acd7e0d5fbf8867cb.jpg
attachments[][type]
  attachment
category
  kemonoparty
content
  Check the attached file!  thank you!!!
count
  1
date
  2017-04-27 22:11:25
edited
  Thu, 27 Apr 2017 22:11:25 GMT
extension
  jpg
file[name]
  Inque_Patreon.jpg
file[path]
  /8d/fb/8dfbc9b141593f71dd89632a09c2f2715af851810ee3ecf21e75909ef715fd35.jpg
filename
  Inque
hash
  c8b963b60373991c8ae6cd49dbfb4a988acaff7f64e5fa0acd7e0d5fbf8867cb
id
  995376
name
  Inque.jpg
num
  1
path
  /c8/b9/c8b963b60373991c8ae6cd49dbfb4a988acaff7f64e5fa0acd7e0d5fbf8867cb.jpg
published
  Thu, 27 Apr 2017 22:11:25 GMT
service
  patreon
shared_file
  False
subcategory
  patreon
title
  [High Res File] Inque from Batman Beyond Patreon Commission.
type
  attachment
user
  402938
username
  reiq
mikf commented 2 years ago

@Skyofflad Post IDs are fixed with commit https://github.com/mikf/gallery-dl/commit/779e75c6f8a21903a1eca1a277b5780a2869730d. I didn't realize files/attachments could have an ID themselves. In my tests everything only had the usual name and path. Thanks for reporting this.

Regarding the 403 issue: Downloading seems to work again when sending a Referer header. gallery-dl -o headers.Referer="https://kemono.party/" ...