Nandaka / PixivUtil2

Download images from Pixiv and more!
http://nandaka.devnull.zone/
BSD 2-Clause "Simplified" License
2.36k stars 257 forks source link

[Feature request] Ugoira only download option #726

Closed pxssy closed 4 years ago

pxssy commented 4 years ago

Prerequisites

Description

When downloading by tags, there is an option to pick either "a"ll content, or "i"llustrations and ugoira or just "m"anga.

Steps to Reproduce

Input: 3 Tags: (アークナイツ or arknights or 明日方舟 or 명일방주) (うごイラ or Ugoira or 動圖 or 우고이라 or 动图 or gif or 3D) Bookmark Count: 0 Use Partial Match (s_tag) [y/n]: y Oldest first[y/n]: Start Page (default=1): End Page (default=0, 0 for no limit): Start Date [YYYY-MM-DD]: End Date [YYYY-MM-DD]: Search type [a-all|i-Illustration and Ugoira|m-manga: u Valid values are 'a', 'i', or 'm'. Search type [a-all|i-Illustration and Ugoira|m-manga: i Reading Arknights\config.ini ... done. Searching for: ((アークナイツ or arknights or 明日方舟 or 명일방주) (うごイラ or Ugoira or 動圖 or 우고이라 or 动图 or gif or 3D)) %28%E3%82%A2%E3%83%BC%E3%82%AF%E3%83%8A%E3%82%A4%E3%83%84%20or%20arknights%20or%20%E6%98%8E%E6%97%A5%E6%96%B9%E8%88%9F%20or%20%EB%AA%85%EC%9D%BC%EB%B0%A9%EC%A3%BC%29%20%28%E3%81%86%E3%81%94%E3%82%A4%E3%83%A9%20or%20Ugoira%20or%20%E5%8B%95%E5%9C%96%20or%20%EC%9A%B0%EA%B3%A0%EC%9D%B4%EB%9D%BC%20or%20%E5%8A%A8%E5%9B%BE%20or%20gif%20or%203D%29 Using Partial Match (s_tag) Looping... for https://www.pixiv.net/ajax/search/artworks/%28%E3%82%A2%E3%83%BC%E3%82%AF%E3%83%8A%E3%82%A4%E3%83%84%20or%20arknights%20or%20%E6%98%8E%E6%97%A5%E6%96%B9%E8%88%9F%20or%20%EB%AA%85%EC%9D%BC%EB%B0%A9%EC%A3%BC%29%20%28%E3%81%86%E3%81%94%E3%82%A4%E3%83%A9%20or%20Ugoira%20or%20%E5%8B%95%E5%9C%96%20or%20%EC%9A%B0%EA%B3%A0%EC%9D%B4%EB%9D%BC%20or%20%E5%8A%A8%E5%9B%BE%20or%20gif%20or%203D%29?word=%28%E3%82%A2%E3%83%BC%E3%82%AF%E3%83%8A%E3%82%A4%E3%83%84%20or%20arknights%20or%20%E6%98%8E%E6%97%A5%E6%96%B9%E8%88%9F%20or%20%EB%AA%85%EC%9D%BC%EB%B0%A9%EC%A3%BC%29%20%28%E3%81%86%E3%81%94%E3%82%A4%E3%83%A9%20or%20Ugoira%20or%20%E5%8B%95%E5%9C%96%20or%20%EC%9A%B0%EA%B3%A0%EC%9D%B4%EB%9D%BC%20or%20%E5%8A%A8%E5%9B%BE%20or%20gif%20or%203D%29&type=illust_and_ugoira

Expected behavior: [What you expected to happen]

With the tags given, ie (うごイラ or Ugoira or 動圖 or 우고이라 or 动图 or gif) i should only expect ugoira content. However, not all Ugoira posts are tagged with those ugoira related tags. I did notice however searching for that string of tags on pixiv itself, an option of only Ugoira posts produces a link like

https://www.pixiv.net/en/tags/(%E3%82%A2%E3%83%BC%E3%82%AF%E3%83%8A%E3%82%A4%E3%83%84%20or%20arknights%20or%20%E6%98%8E%E6%97%A5%E6%96%B9%E8%88%9F%20or%20%EB%AA%85%EC%9D%BC%EB%B0%A9%EC%A3%BC)%20(%E3%81%86%E3%81%94%E3%82%A4%E3%83%A9%20or%20Ugoira%20or%20%E5%8B%95%E5%9C%96%20or%20%EC%9A%B0%EA%B3%A0%EC%9D%B4%EB%9D%BC%20or%20%E5%8A%A8%E5%9B%BE%20or%20gif%20or%203D)/illustrations?s_mode=s_tag&type=ugoira

is available.

"&type=ugoira" should therefore produce ONLY ugoira instead of illustrations AND ugoira like "&type=illust_and_ugoira"

tried u in place of i does not work. Trying to capture every possibility in every language of "ugoira" like using tags (うごイラ or Ugoira or 動圖 or 우고이라 or 动图 or gif) is very tedious and still doesn't cover all ground. I hope you can implement this

Nandaka commented 4 years ago

well, the server itself doesn't provide the filtering function from the API, so technically it is not possible unless it goes all the way to the image details page and check if the provided links contains ugoira or not.

pxssy commented 4 years ago

Oh i thought because of the link that appears like https://www.pixiv.net/ajax/search/artworks/%28%E3%82%A2%E3%83%BC%E3%82%AF%E3%83%8A%E3%82%A4%E3%83%84%20or%20arknights%20or%20%E6%98%8E%E6%97%A5%E6%96%B9%E8%88%9F%20or%20%EB%AA%85%EC%9D%BC%EB%B0%A9%EC%A3%BC%29%20%28%E3%81%86%E3%81%94%E3%82%A4%E3%83%A9%20or%20Ugoira%20or%20%E5%8B%95%E5%9C%96%20or%20%EC%9A%B0%EA%B3%A0%EC%9D%B4%EB%9D%BC%20or%20%E5%8A%A8%E5%9B%BE%20or%20gif%20or%203D%29?word=%28%E3%82%A2%E3%83%BC%E3%82%AF%E3%83%8A%E3%82%A4%E3%83%84%20or%20arknights%20or%20%E6%98%8E%E6%97%A5%E6%96%B9%E8%88%9F%20or%20%EB%AA%85%EC%9D%BC%EB%B0%A9%EC%A3%BC%29%20%28%E3%81%86%E3%81%94%E3%82%A4%E3%83%A9%20or%20Ugoira%20or%20%E5%8B%95%E5%9C%96%20or%20%EC%9A%B0%EA%B3%A0%EC%9D%B4%EB%9D%BC%20or%20%E5%8A%A8%E5%9B%BE%20or%20gif%20or%203D%29&type=illust_and_ugoira you were using the site link and could just edit it the 'illust_and_ugoira" to just "ugoira" . I'm not too sure about API but i guess that's a downer if it can't be done from API.

Honestly i've already downloaded the ugoira zip and json, but for a very long time my pc wasn't very powerful and it was quite slow to encode zip into ugoira, so i disabled it. Now what i've upgraded it somewhat, i want the ugoira.

Any idea how i can create those ugoira/webm now with my downloaded zips and json? I can't seem to find any source on how to do this.

Nandaka commented 4 years ago

Assuming the configuration: overwrite, alwaysCheckFileSize, and createUgoira are set to True. Assuming you didn't change the filename format and the files still there, it should try to convert the existing files.

https://github.com/Nandaka/PixivUtil2/blob/master/PixivDownloadHandler.py#L81

else, if you are running from source code, you can call the method directly from python console: https://github.com/Nandaka/PixivUtil2/blob/e550cee0e0ce6ed72444b342db1b6800ef361efc/PixivDownloadHandler.py#L332

xuejianxianzun commented 4 years ago

Hello! Can I recommend my downloader? On all pages, you can set the type of work: Illustrations Manga Ugoira Novel https://github.com/xuejianxianzun/PixivBatchDownloader

Nandaka commented 4 years ago

added in latest commit only for batch job, refer to https://github.com/Nandaka/PixivUtil2/wiki/Using-Batch-Job-(Experimental) for sample batch job format