Jeffchen54 / KMP-Kemono-Downloader

Simple Kemono downloader
MIT License
47 stars 4 forks source link

Maybe a bug #15

Open mario6714 opened 1 year ago

mario6714 commented 1 year ago

Is working since 2:29 pm and the actual hour is 08:14 pm

imagen

KMPDownloader.py -r 1 -v -u -o -m -b -t 2 -d D:\KMPDownloader

https://kemono.party/fantia/user/6817

D:\KMPDownloader>KMPDownloader.py -r 1 -v -u -o -m -b -t 2 -d D:\KMPDownloader 04/03/2023 02:17:00 PM RETRIES -> 1 04/03/2023 02:17:00 PM UNZIP -> True 04/03/2023 02:17:00 PM UNPACKED -> True 04/03/2023 02:17:00 PM OMITCOMMENTS -> True 04/03/2023 02:17:00 PM OMITPOSTCONTENT -> True 04/03/2023 02:17:00 PM TRACK -> True 04/03/2023 02:17:00 PM DOWNLOAD_THREAD_COUNT -> 2 04/03/2023 02:17:00 PM FOLDER -> D:\KMPDownloader\ 04/03/2023 02:17:00 PM Please wait, scanning directory 04/03/2023 02:17:04 PM Finished scanning directory Input a url, or type 'quit' to exit> https://kemono.party/fantia/user/6817 04/03/2023 02:17:22 PM Finished scanning https://kemono.party/fantia/user/6817/post/1612715 04/03/2023 02:17:23 PM Finished scanning https://kemono.party/fantia/user/6817/post/1592278 04/03/2023 02:17:25 PM Finished scanning https://kemono.party/fantia/user/6817/post/1572847 04/03/2023 02:17:26 PM Finished scanning https://kemono.party/fantia/user/6817/post/1568792 04/03/2023 02:17:28 PM Finished scanning https://kemono.party/fantia/user/6817/post/1550770 04/03/2023 02:17:29 PM Finished scanning https://kemono.party/fantia/user/6817/post/1543570 04/03/2023 02:17:30 PM Finished scanning https://kemono.party/fantia/user/6817/post/1535300 04/03/2023 02:17:31 PM Finished scanning https://kemono.party/fantia/user/6817/post/1497588 04/03/2023 02:17:32 PM Finished scanning https://kemono.party/fantia/user/6817/post/1488998 04/03/2023 02:17:33 PM Finished scanning https://kemono.party/fantia/user/6817/post/1473887 04/03/2023 02:17:34 PM Finished scanning https://kemono.party/fantia/user/6817/post/1454285 04/03/2023 02:17:35 PM Finished scanning https://kemono.party/fantia/user/6817/post/1437261 04/03/2023 02:17:36 PM Finished scanning https://kemono.party/fantia/user/6817/post/1433749 04/03/2023 02:17:37 PM Finished scanning https://kemono.party/fantia/user/6817/post/1425596 04/03/2023 02:17:38 PM Finished scanning https://kemono.party/fantia/user/6817/post/1408703 04/03/2023 02:17:39 PM Finished scanning https://kemono.party/fantia/user/6817/post/1402772 04/03/2023 02:17:40 PM Finished scanning https://kemono.party/fantia/user/6817/post/1400989 04/03/2023 02:17:41 PM Finished scanning https://kemono.party/fantia/user/6817/post/1398281 04/03/2023 02:17:42 PM Finished scanning https://kemono.party/fantia/user/6817/post/1389456 04/03/2023 02:17:43 PM Finished scanning https://kemono.party/fantia/user/6817/post/1363337 04/03/2023 02:17:44 PM Finished scanning https://kemono.party/fantia/user/6817/post/1356488 04/03/2023 02:17:45 PM Finished scanning https://kemono.party/fantia/user/6817/post/1324726 04/03/2023 02:17:46 PM Finished scanning https://kemono.party/fantia/user/6817/post/1248920 04/03/2023 02:17:47 PM Finished scanning https://kemono.party/fantia/user/6817/post/1235054 04/03/2023 02:17:48 PM Finished scanning https://kemono.party/fantia/user/6817/post/1223850 04/03/2023 02:17:49 PM Finished scanning https://kemono.party/fantia/user/6817/post/1223848 04/03/2023 02:17:50 PM Finished scanning https://kemono.party/fantia/user/6817/post/1193673 04/03/2023 02:17:51 PM Finished scanning https://kemono.party/fantia/user/6817/post/1189042 04/03/2023 02:17:52 PM Finished scanning https://kemono.party/fantia/user/6817/post/1187155 04/03/2023 02:17:53 PM Finished scanning https://kemono.party/fantia/user/6817/post/1183150 04/03/2023 02:17:54 PM Finished scanning https://kemono.party/fantia/user/6817/post/1172336 04/03/2023 02:17:55 PM Finished scanning https://kemono.party/fantia/user/6817/post/1157957 04/03/2023 02:17:56 PM Finished scanning https://kemono.party/fantia/user/6817/post/1130118 04/03/2023 02:17:57 PM Finished scanning https://kemono.party/fantia/user/6817/post/1126581 04/03/2023 02:17:58 PM Finished scanning https://kemono.party/fantia/user/6817/post/1118641 04/03/2023 02:17:59 PM Finished scanning https://kemono.party/fantia/user/6817/post/1104059 04/03/2023 02:18:00 PM Finished scanning https://kemono.party/fantia/user/6817/post/1091990 04/03/2023 02:18:01 PM Finished scanning https://kemono.party/fantia/user/6817/post/1058411 04/03/2023 02:18:02 PM Finished scanning https://kemono.party/fantia/user/6817/post/1049272 04/03/2023 02:18:03 PM Finished scanning https://kemono.party/fantia/user/6817/post/1039535 04/03/2023 02:18:04 PM Finished scanning https://kemono.party/fantia/user/6817/post/1033995 04/03/2023 02:18:05 PM Finished scanning https://kemono.party/fantia/user/6817/post/1010952 04/03/2023 02:18:06 PM Finished scanning https://kemono.party/fantia/user/6817/post/1002784 04/03/2023 02:18:07 PM Finished scanning https://kemono.party/fantia/user/6817/post/1002773 04/03/2023 02:18:08 PM Finished scanning https://kemono.party/fantia/user/6817/post/998643 04/03/2023 02:18:09 PM Finished scanning https://kemono.party/fantia/user/6817/post/991488 04/03/2023 02:18:10 PM Finished scanning https://kemono.party/fantia/user/6817/post/987148 04/03/2023 02:18:11 PM Finished scanning https://kemono.party/fantia/user/6817/post/976329 04/03/2023 02:18:12 PM Finished scanning https://kemono.party/fantia/user/6817/post/967635 04/03/2023 02:18:13 PM Finished scanning https://kemono.party/fantia/user/6817/post/932987 04/03/2023 02:18:14 PM Finished scanning https://kemono.party/fantia/user/6817/post/928607 04/03/2023 02:18:15 PM Finished scanning https://kemono.party/fantia/user/6817/post/919630 04/03/2023 02:18:16 PM Finished scanning https://kemono.party/fantia/user/6817/post/912147 04/03/2023 02:18:17 PM Finished scanning https://kemono.party/fantia/user/6817/post/907899 04/03/2023 02:18:18 PM Finished scanning https://kemono.party/fantia/user/6817/post/906128 04/03/2023 02:18:19 PM Finished scanning https://kemono.party/fantia/user/6817/post/885624 04/03/2023 02:18:20 PM Finished scanning https://kemono.party/fantia/user/6817/post/863856 04/03/2023 02:18:21 PM Finished scanning https://kemono.party/fantia/user/6817/post/857904 04/03/2023 02:18:22 PM Finished scanning https://kemono.party/fantia/user/6817/post/829319 04/03/2023 02:18:23 PM Finished scanning https://kemono.party/fantia/user/6817/post/816121 04/03/2023 02:18:24 PM Finished scanning https://kemono.party/fantia/user/6817/post/798604 04/03/2023 02:18:25 PM Finished scanning https://kemono.party/fantia/user/6817/post/790387 04/03/2023 02:18:26 PM Finished scanning https://kemono.party/fantia/user/6817/post/744495 04/03/2023 02:18:27 PM Finished scanning https://kemono.party/fantia/user/6817/post/734912 04/03/2023 02:18:28 PM Finished scanning https://kemono.party/fantia/user/6817/post/722693 04/03/2023 02:18:29 PM Finished scanning https://kemono.party/fantia/user/6817/post/708269 04/03/2023 02:18:30 PM Finished scanning https://kemono.party/fantia/user/6817/post/717733 04/03/2023 02:18:31 PM Finished scanning https://kemono.party/fantia/user/6817/post/675748 04/03/2023 02:18:32 PM Finished scanning https://kemono.party/fantia/user/6817/post/664289 04/03/2023 02:18:33 PM Finished scanning https://kemono.party/fantia/user/6817/post/657356 04/03/2023 02:18:34 PM Finished scanning https://kemono.party/fantia/user/6817/post/649719 04/03/2023 02:18:35 PM Finished scanning https://kemono.party/fantia/user/6817/post/645999 04/03/2023 02:18:36 PM Finished scanning https://kemono.party/fantia/user/6817/post/615664 04/03/2023 02:18:37 PM Finished scanning https://kemono.party/fantia/user/6817/post/609423 04/03/2023 02:18:38 PM Finished scanning https://kemono.party/fantia/user/6817/post/595061 04/03/2023 02:18:39 PM Finished scanning https://kemono.party/fantia/user/6817/post/588691 04/03/2023 02:18:40 PM Finished scanning https://kemono.party/fantia/user/6817/post/583253 04/03/2023 02:18:41 PM Finished scanning https://kemono.party/fantia/user/6817/post/574534 04/03/2023 02:18:42 PM Finished scanning https://kemono.party/fantia/user/6817/post/563797 04/03/2023 02:18:43 PM Finished scanning https://kemono.party/fantia/user/6817/post/563792 04/03/2023 02:18:44 PM Finished scanning https://kemono.party/fantia/user/6817/post/563786 04/03/2023 02:18:45 PM Finished scanning https://kemono.party/fantia/user/6817/post/509304 04/03/2023 02:18:46 PM Finished scanning https://kemono.party/fantia/user/6817/post/491152 04/03/2023 02:18:47 PM Finished scanning https://kemono.party/fantia/user/6817/post/480860 04/03/2023 02:18:48 PM Finished scanning https://kemono.party/fantia/user/6817/post/453554 04/03/2023 02:18:49 PM Finished scanning https://kemono.party/fantia/user/6817/post/446354 04/03/2023 02:18:50 PM Finished scanning https://kemono.party/fantia/user/6817/post/439376 04/03/2023 02:18:51 PM Finished scanning https://kemono.party/fantia/user/6817/post/421293 04/03/2023 02:18:52 PM Finished scanning https://kemono.party/fantia/user/6817/post/409504 04/03/2023 02:18:53 PM Finished scanning https://kemono.party/fantia/user/6817/post/402913 04/03/2023 02:18:54 PM Finished scanning https://kemono.party/fantia/user/6817/post/398973 04/03/2023 02:18:55 PM Finished scanning https://kemono.party/fantia/user/6817/post/381097 04/03/2023 02:18:56 PM Finished scanning https://kemono.party/fantia/user/6817/post/377724 04/03/2023 02:18:57 PM Finished scanning https://kemono.party/fantia/user/6817/post/372187 04/03/2023 02:18:58 PM Finished scanning https://kemono.party/fantia/user/6817/post/349773 04/03/2023 02:18:59 PM Finished scanning https://kemono.party/fantia/user/6817/post/344861 04/03/2023 02:19:00 PM Finished scanning https://kemono.party/fantia/user/6817/post/324624 04/03/2023 02:19:01 PM Finished scanning https://kemono.party/fantia/user/6817/post/310315 04/03/2023 02:19:02 PM Finished scanning https://kemono.party/fantia/user/6817/post/302836 04/03/2023 02:19:03 PM Finished scanning https://kemono.party/fantia/user/6817/post/296244 04/03/2023 02:19:04 PM Finished scanning https://kemono.party/fantia/user/6817/post/287394 04/03/2023 02:19:05 PM Finished scanning https://kemono.party/fantia/user/6817/post/277843 04/03/2023 02:19:06 PM Finished scanning https://kemono.party/fantia/user/6817/post/260103 04/03/2023 02:19:07 PM Finished scanning https://kemono.party/fantia/user/6817/post/258836 04/03/2023 02:19:08 PM Finished scanning https://kemono.party/fantia/user/6817/post/252511 04/03/2023 02:19:09 PM Finished scanning https://kemono.party/fantia/user/6817/post/241428 04/03/2023 02:19:10 PM Finished scanning https://kemono.party/fantia/user/6817/post/230775 04/03/2023 02:19:11 PM Finished scanning https://kemono.party/fantia/user/6817/post/228780 04/03/2023 02:19:12 PM Finished scanning https://kemono.party/fantia/user/6817/post/225111 04/03/2023 02:19:13 PM Finished scanning https://kemono.party/fantia/user/6817/post/216164 04/03/2023 02:19:14 PM Finished scanning https://kemono.party/fantia/user/6817/post/211374 04/03/2023 02:19:15 PM Finished scanning https://kemono.party/fantia/user/6817/post/206046 04/03/2023 02:19:16 PM Finished scanning https://kemono.party/fantia/user/6817/post/203005 04/03/2023 02:19:17 PM Finished scanning https://kemono.party/fantia/user/6817/post/197080 04/03/2023 02:19:18 PM Finished scanning https://kemono.party/fantia/user/6817/post/192111 04/03/2023 02:19:19 PM Finished scanning https://kemono.party/fantia/user/6817/post/185867 04/03/2023 02:19:20 PM Finished scanning https://kemono.party/fantia/user/6817/post/181699 04/03/2023 02:19:21 PM Finished scanning https://kemono.party/fantia/user/6817/post/176408 04/03/2023 02:19:22 PM Finished scanning https://kemono.party/fantia/user/6817/post/173018 04/03/2023 02:19:23 PM Finished scanning https://kemono.party/fantia/user/6817/post/171908

mario6714 commented 10 months ago

One question , this speed is normal?

imagen

I just run a speed test in speedtest

imagen

Is the site "Kemono.party" right?

Jeffchen54 commented 10 months ago

Yeah, even though your internet is fast, kemono slows down downloading so you won't be able to get anywhere as fast downloads from the site. The site is also quite slow after the issue it had earlier today. It took me around 5 seconds loading a 149kb file on a web browser so the site has an issue somewhere along its pipeline.

mario6714 commented 9 months ago

Hi, one question, i want to use "SERVER_NAME_DOWNLOAD" but still use the other format:

imagen

and for example, this post has the same image 2 times:

imagen

Is normal have a duplicate?

Jeffchen54 commented 9 months ago

The server fname switch isn't compatible with the latest update, you'll have to use a previous release for it to work. I don't think it's compatible with the last update as well due to the --date switch too.

mario6714 commented 9 months ago

and about the duplicates?

Jeffchen54 commented 9 months ago

You'll have dupes with the fname switch because it isn't compatible. Remove it and it'll work.

mario6714 commented 9 months ago

a ok

mario6714 commented 9 months ago

-e?

mario6714 commented 9 months ago

like this work?

KMPDownloader.py -v -w -u -o -m -b -t 2 -d D:\KMPDownloader --RENAME -r 4

Jeffchen54 commented 9 months ago

I don't recall specifically which switch it is but don't use the switch for server fname. If it doesn't say Server fname is true when you start up the program, you'll be fine.

mario6714 commented 9 months ago

Like this ?

imagen

Jeffchen54 commented 9 months ago

Yeah you shouldn't get any duplicates with that config.

mario6714 commented 9 months ago

thanks

Jeffchen54 commented 9 months ago

No problem, I'll look into getting the switch to work in a future update.

mario6714 commented 9 months ago

still duplicates:

imagen

Jeffchen54 commented 9 months ago

I found out why, those aren't duplicates. If you go onto the work page, there are 5 images there which matches up with what you are having. The issue is on Kemono importer for importing duplicate images onto the page.

mario6714 commented 9 months ago

i way to avoid this?

imagen

Jeffchen54 commented 9 months ago

Isn't possible unfortunately, the site slows down downloads by imposing them. You can try a VPN but it isn't guaranteed you can avoid them.

mario6714 commented 9 months ago

Hi, one question, is not the same link?

imagen

429 TIMEOUT -> SRC: https://c3.kemono.party/data/d2/56/d256c2768350c504d422635e93913843f724e5278f36f3df751113432ae22591.png?f=IMA%201.png, FNAME: D:\KMPDownloader\Aroon\79983738 ANIME -isekai nonbiri nouka LUCY HD NSFW by Aroon from Patreon Kemono - 2.png 429 TIMEOUT -> SRC: https://c3.kemono.party/data/d2/56/d256c2768350c504d422635e93913843f724e5278f36f3df751113432ae22591.png?f=IMA%201.png, FNAME: D:\KMPDownloader\Aroon\79983738 ANIME -isekai nonbiri nouka LUCY HD NSFW by Aroon from Patreon Kemono - 3.png

Jeffchen54 commented 9 months ago

It might be, the program doesn't use the links for duplicate checking. It only checks the work name and where it appears in the work. Checking src could be implemented in the future but currently, duplicate check in the program only cares if the number of files on your pc matches the number of files shown in kemono.

mario6714 commented 9 months ago

Support in the future for "Kemono.su" for example?

imagen

thanks

mario6714 commented 9 months ago

and the program just stopped work:

imagen

https://kemono.party/fanbox/user/3316400

Error in "Kemono"?

Jeffchen54 commented 9 months ago

Looks like Kemono implemented DDOS guard effectively trying to kill scrapers, I don't think the program will work anymore unless I find a way to bypass it. I'll have to do research into getting around it which will take some time. I know one method is to enter kemono itself and solve the puzzle then run the program though I'm not sure how reliable this approach is going to be in the long run.

Jeffchen54 commented 9 months ago

Support in the future for "Kemono.su" for example?

imagen

thanks

This is already supported, use the --prefix switch so it will be "--prefix "https://kemono.su"". I chose this approach because who knows what domain extensions kemono uses in the future

mario6714 commented 9 months ago

Looks like Kemono implemented DDOS guard effectively trying to kill scrapers, I don't think the program will work anymore unless I find a way to bypass it. I'll have to do research into getting around it which will take some time. I know one method is to enter kemono itself and solve the puzzle then run the program though I'm not sure how reliable this approach is going to be in the long run.

Is working again

mario6714 commented 9 months ago

Support in the future for "Kemono.su" for example? imagen thanks

This is already supported, use the --prefix switch so it will be "--prefix "https://kemono.su"". I chose this approach because who knows what domain extensions kemono uses in the future

Ok, thanks

mario6714 commented 9 months ago

Maybe a new bug

For example this link "https://kemono.party/patreon/user/29542203" only have few posts

But the program is like this and it doesn't download anything:

imagen

https://kemono.party/patreon/user/29542203

KMPDownloader.py -v -u -o -w -m -b -t 2 -d D:\KMPDownloader --RENAME -r 12

any idea?

Jeffchen54 commented 9 months ago

That's odd, the page only shows 8 posts yet the downloader shows more, I'll have to take a good look into what's causing it.

Jeffchen54 commented 9 months ago

It looks like Kemono rolled out changes to how the paging system works along with a recruitment banner looks pretty tempting tbh. It's a quick fix so I should have something ready today.

Jeffchen54 commented 9 months ago

Download this #29 and see if it fixes your issue, it should work from my own testing but the PR as a whole needs to be tested more before I release it.

mario6714 commented 9 months ago

Download this #29 and see if it fixes your issue, it should work from my own testing but the PR as a whole needs to be tested more before I release it.

Ok, let me try, thanks

mario6714 commented 9 months ago

imagen

i just create a new text file in the folder "user-agent.txt"

mario6714 commented 9 months ago

I think is working now, thanks

imagen

mario6714 commented 9 months ago

Is ok like this?

imagen

Jeffchen54 commented 9 months ago

Huh, I thought I uploaded that but forgot, you can paste one of the agents from here: https://www.whatismybrowser.com/guides/the-latest-user-agent/firefox. So it could be like Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:109.0) Gecko/20100101 Firefox/118.0. As long as you have user-agent.txt where KMPDownloader is and paste a agent in, you should be good to go.