Maybe a bug

[mario6714]

Is working since 2:29 pm and the actual hour is 08:14 pm

KMPDownloader.py -r 1 -v -u -o -m -b -t 2 -d D:\KMPDownloader

https://kemono.party/fantia/user/6817

D:\KMPDownloader>KMPDownloader.py -r 1 -v -u -o -m -b -t 2 -d D:\KMPDownloader 04/03/2023 02:17:00 PM RETRIES -> 1 04/03/2023 02:17:00 PM UNZIP -> True 04/03/2023 02:17:00 PM UNPACKED -> True 04/03/2023 02:17:00 PM OMITCOMMENTS -> True 04/03/2023 02:17:00 PM OMITPOSTCONTENT -> True 04/03/2023 02:17:00 PM TRACK -> True 04/03/2023 02:17:00 PM DOWNLOAD_THREAD_COUNT -> 2 04/03/2023 02:17:00 PM FOLDER -> D:\KMPDownloader\ 04/03/2023 02:17:00 PM Please wait, scanning directory 04/03/2023 02:17:04 PM Finished scanning directory Input a url, or type 'quit' to exit> https://kemono.party/fantia/user/6817 04/03/2023 02:17:22 PM Finished scanning https://kemono.party/fantia/user/6817/post/1612715 04/03/2023 02:17:23 PM Finished scanning https://kemono.party/fantia/user/6817/post/1592278 04/03/2023 02:17:25 PM Finished scanning https://kemono.party/fantia/user/6817/post/1572847 04/03/2023 02:17:26 PM Finished scanning https://kemono.party/fantia/user/6817/post/1568792 04/03/2023 02:17:28 PM Finished scanning https://kemono.party/fantia/user/6817/post/1550770 04/03/2023 02:17:29 PM Finished scanning https://kemono.party/fantia/user/6817/post/1543570 04/03/2023 02:17:30 PM Finished scanning https://kemono.party/fantia/user/6817/post/1535300 04/03/2023 02:17:31 PM Finished scanning https://kemono.party/fantia/user/6817/post/1497588 04/03/2023 02:17:32 PM Finished scanning https://kemono.party/fantia/user/6817/post/1488998 04/03/2023 02:17:33 PM Finished scanning https://kemono.party/fantia/user/6817/post/1473887 04/03/2023 02:17:34 PM Finished scanning https://kemono.party/fantia/user/6817/post/1454285 04/03/2023 02:17:35 PM Finished scanning https://kemono.party/fantia/user/6817/post/1437261 04/03/2023 02:17:36 PM Finished scanning https://kemono.party/fantia/user/6817/post/1433749 04/03/2023 02:17:37 PM Finished scanning https://kemono.party/fantia/user/6817/post/1425596 04/03/2023 02:17:38 PM Finished scanning https://kemono.party/fantia/user/6817/post/1408703 04/03/2023 02:17:39 PM Finished scanning https://kemono.party/fantia/user/6817/post/1402772 04/03/2023 02:17:40 PM Finished scanning https://kemono.party/fantia/user/6817/post/1400989 04/03/2023 02:17:41 PM Finished scanning https://kemono.party/fantia/user/6817/post/1398281 04/03/2023 02:17:42 PM Finished scanning https://kemono.party/fantia/user/6817/post/1389456 04/03/2023 02:17:43 PM Finished scanning https://kemono.party/fantia/user/6817/post/1363337 04/03/2023 02:17:44 PM Finished scanning https://kemono.party/fantia/user/6817/post/1356488 04/03/2023 02:17:45 PM Finished scanning https://kemono.party/fantia/user/6817/post/1324726 04/03/2023 02:17:46 PM Finished scanning https://kemono.party/fantia/user/6817/post/1248920 04/03/2023 02:17:47 PM Finished scanning https://kemono.party/fantia/user/6817/post/1235054 04/03/2023 02:17:48 PM Finished scanning https://kemono.party/fantia/user/6817/post/1223850 04/03/2023 02:17:49 PM Finished scanning https://kemono.party/fantia/user/6817/post/1223848 04/03/2023 02:17:50 PM Finished scanning https://kemono.party/fantia/user/6817/post/1193673 04/03/2023 02:17:51 PM Finished scanning https://kemono.party/fantia/user/6817/post/1189042 04/03/2023 02:17:52 PM Finished scanning https://kemono.party/fantia/user/6817/post/1187155 04/03/2023 02:17:53 PM Finished scanning https://kemono.party/fantia/user/6817/post/1183150 04/03/2023 02:17:54 PM Finished scanning https://kemono.party/fantia/user/6817/post/1172336 04/03/2023 02:17:55 PM Finished scanning https://kemono.party/fantia/user/6817/post/1157957 04/03/2023 02:17:56 PM Finished scanning https://kemono.party/fantia/user/6817/post/1130118 04/03/2023 02:17:57 PM Finished scanning https://kemono.party/fantia/user/6817/post/1126581 04/03/2023 02:17:58 PM Finished scanning https://kemono.party/fantia/user/6817/post/1118641 04/03/2023 02:17:59 PM Finished scanning https://kemono.party/fantia/user/6817/post/1104059 04/03/2023 02:18:00 PM Finished scanning https://kemono.party/fantia/user/6817/post/1091990 04/03/2023 02:18:01 PM Finished scanning https://kemono.party/fantia/user/6817/post/1058411 04/03/2023 02:18:02 PM Finished scanning https://kemono.party/fantia/user/6817/post/1049272 04/03/2023 02:18:03 PM Finished scanning https://kemono.party/fantia/user/6817/post/1039535 04/03/2023 02:18:04 PM Finished scanning https://kemono.party/fantia/user/6817/post/1033995 04/03/2023 02:18:05 PM Finished scanning https://kemono.party/fantia/user/6817/post/1010952 04/03/2023 02:18:06 PM Finished scanning https://kemono.party/fantia/user/6817/post/1002784 04/03/2023 02:18:07 PM Finished scanning https://kemono.party/fantia/user/6817/post/1002773 04/03/2023 02:18:08 PM Finished scanning https://kemono.party/fantia/user/6817/post/998643 04/03/2023 02:18:09 PM Finished scanning https://kemono.party/fantia/user/6817/post/991488 04/03/2023 02:18:10 PM Finished scanning https://kemono.party/fantia/user/6817/post/987148 04/03/2023 02:18:11 PM Finished scanning https://kemono.party/fantia/user/6817/post/976329 04/03/2023 02:18:12 PM Finished scanning https://kemono.party/fantia/user/6817/post/967635 04/03/2023 02:18:13 PM Finished scanning https://kemono.party/fantia/user/6817/post/932987 04/03/2023 02:18:14 PM Finished scanning https://kemono.party/fantia/user/6817/post/928607 04/03/2023 02:18:15 PM Finished scanning https://kemono.party/fantia/user/6817/post/919630 04/03/2023 02:18:16 PM Finished scanning https://kemono.party/fantia/user/6817/post/912147 04/03/2023 02:18:17 PM Finished scanning https://kemono.party/fantia/user/6817/post/907899 04/03/2023 02:18:18 PM Finished scanning https://kemono.party/fantia/user/6817/post/906128 04/03/2023 02:18:19 PM Finished scanning https://kemono.party/fantia/user/6817/post/885624 04/03/2023 02:18:20 PM Finished scanning https://kemono.party/fantia/user/6817/post/863856 04/03/2023 02:18:21 PM Finished scanning https://kemono.party/fantia/user/6817/post/857904 04/03/2023 02:18:22 PM Finished scanning https://kemono.party/fantia/user/6817/post/829319 04/03/2023 02:18:23 PM Finished scanning https://kemono.party/fantia/user/6817/post/816121 04/03/2023 02:18:24 PM Finished scanning https://kemono.party/fantia/user/6817/post/798604 04/03/2023 02:18:25 PM Finished scanning https://kemono.party/fantia/user/6817/post/790387 04/03/2023 02:18:26 PM Finished scanning https://kemono.party/fantia/user/6817/post/744495 04/03/2023 02:18:27 PM Finished scanning https://kemono.party/fantia/user/6817/post/734912 04/03/2023 02:18:28 PM Finished scanning https://kemono.party/fantia/user/6817/post/722693 04/03/2023 02:18:29 PM Finished scanning https://kemono.party/fantia/user/6817/post/708269 04/03/2023 02:18:30 PM Finished scanning https://kemono.party/fantia/user/6817/post/717733 04/03/2023 02:18:31 PM Finished scanning https://kemono.party/fantia/user/6817/post/675748 04/03/2023 02:18:32 PM Finished scanning https://kemono.party/fantia/user/6817/post/664289 04/03/2023 02:18:33 PM Finished scanning https://kemono.party/fantia/user/6817/post/657356 04/03/2023 02:18:34 PM Finished scanning https://kemono.party/fantia/user/6817/post/649719 04/03/2023 02:18:35 PM Finished scanning https://kemono.party/fantia/user/6817/post/645999 04/03/2023 02:18:36 PM Finished scanning https://kemono.party/fantia/user/6817/post/615664 04/03/2023 02:18:37 PM Finished scanning https://kemono.party/fantia/user/6817/post/609423 04/03/2023 02:18:38 PM Finished scanning https://kemono.party/fantia/user/6817/post/595061 04/03/2023 02:18:39 PM Finished scanning https://kemono.party/fantia/user/6817/post/588691 04/03/2023 02:18:40 PM Finished scanning https://kemono.party/fantia/user/6817/post/583253 04/03/2023 02:18:41 PM Finished scanning https://kemono.party/fantia/user/6817/post/574534 04/03/2023 02:18:42 PM Finished scanning https://kemono.party/fantia/user/6817/post/563797 04/03/2023 02:18:43 PM Finished scanning https://kemono.party/fantia/user/6817/post/563792 04/03/2023 02:18:44 PM Finished scanning https://kemono.party/fantia/user/6817/post/563786 04/03/2023 02:18:45 PM Finished scanning https://kemono.party/fantia/user/6817/post/509304 04/03/2023 02:18:46 PM Finished scanning https://kemono.party/fantia/user/6817/post/491152 04/03/2023 02:18:47 PM Finished scanning https://kemono.party/fantia/user/6817/post/480860 04/03/2023 02:18:48 PM Finished scanning https://kemono.party/fantia/user/6817/post/453554 04/03/2023 02:18:49 PM Finished scanning https://kemono.party/fantia/user/6817/post/446354 04/03/2023 02:18:50 PM Finished scanning https://kemono.party/fantia/user/6817/post/439376 04/03/2023 02:18:51 PM Finished scanning https://kemono.party/fantia/user/6817/post/421293 04/03/2023 02:18:52 PM Finished scanning https://kemono.party/fantia/user/6817/post/409504 04/03/2023 02:18:53 PM Finished scanning https://kemono.party/fantia/user/6817/post/402913 04/03/2023 02:18:54 PM Finished scanning https://kemono.party/fantia/user/6817/post/398973 04/03/2023 02:18:55 PM Finished scanning https://kemono.party/fantia/user/6817/post/381097 04/03/2023 02:18:56 PM Finished scanning https://kemono.party/fantia/user/6817/post/377724 04/03/2023 02:18:57 PM Finished scanning https://kemono.party/fantia/user/6817/post/372187 04/03/2023 02:18:58 PM Finished scanning https://kemono.party/fantia/user/6817/post/349773 04/03/2023 02:18:59 PM Finished scanning https://kemono.party/fantia/user/6817/post/344861 04/03/2023 02:19:00 PM Finished scanning https://kemono.party/fantia/user/6817/post/324624 04/03/2023 02:19:01 PM Finished scanning https://kemono.party/fantia/user/6817/post/310315 04/03/2023 02:19:02 PM Finished scanning https://kemono.party/fantia/user/6817/post/302836 04/03/2023 02:19:03 PM Finished scanning https://kemono.party/fantia/user/6817/post/296244 04/03/2023 02:19:04 PM Finished scanning https://kemono.party/fantia/user/6817/post/287394 04/03/2023 02:19:05 PM Finished scanning https://kemono.party/fantia/user/6817/post/277843 04/03/2023 02:19:06 PM Finished scanning https://kemono.party/fantia/user/6817/post/260103 04/03/2023 02:19:07 PM Finished scanning https://kemono.party/fantia/user/6817/post/258836 04/03/2023 02:19:08 PM Finished scanning https://kemono.party/fantia/user/6817/post/252511 04/03/2023 02:19:09 PM Finished scanning https://kemono.party/fantia/user/6817/post/241428 04/03/2023 02:19:10 PM Finished scanning https://kemono.party/fantia/user/6817/post/230775 04/03/2023 02:19:11 PM Finished scanning https://kemono.party/fantia/user/6817/post/228780 04/03/2023 02:19:12 PM Finished scanning https://kemono.party/fantia/user/6817/post/225111 04/03/2023 02:19:13 PM Finished scanning https://kemono.party/fantia/user/6817/post/216164 04/03/2023 02:19:14 PM Finished scanning https://kemono.party/fantia/user/6817/post/211374 04/03/2023 02:19:15 PM Finished scanning https://kemono.party/fantia/user/6817/post/206046 04/03/2023 02:19:16 PM Finished scanning https://kemono.party/fantia/user/6817/post/203005 04/03/2023 02:19:17 PM Finished scanning https://kemono.party/fantia/user/6817/post/197080 04/03/2023 02:19:18 PM Finished scanning https://kemono.party/fantia/user/6817/post/192111 04/03/2023 02:19:19 PM Finished scanning https://kemono.party/fantia/user/6817/post/185867 04/03/2023 02:19:20 PM Finished scanning https://kemono.party/fantia/user/6817/post/181699 04/03/2023 02:19:21 PM Finished scanning https://kemono.party/fantia/user/6817/post/176408 04/03/2023 02:19:22 PM Finished scanning https://kemono.party/fantia/user/6817/post/173018 04/03/2023 02:19:23 PM Finished scanning https://kemono.party/fantia/user/6817/post/171908

[Jeffchen54]

I will be a little busy for the next few weeks but I'll see what I can do to fix this issue. Upload your database file and I'll have a look at it as soon as I can.

[mario6714]

I remove the "track" and the same problem:

D:\KMPDownloader>KMPDownloader.py -v -u -o -m -t 2 -d D:\KMPDownloader

[mario6714]

Only happen in here:

https://kemono.party/fantia/user/6817

[mario6714]

I found a folder inside where i can't access, i just deleted the root folder "artist" and going to try again

[Jeffchen54]

From a preliminary observation, it seems like the resources on that specific artist's page aren't available at the moment. The best solution right now is to not download from that specific artist until it is available again. I'm unable to access some items even without using the downloader like here. I will look into looking at skipping these kinds of issues in a later update.

[mario6714]

Ok, thanks, want to make this as closed?

[Jeffchen54]

I'd suggest keeping it open since there is a change I would like to implement because of this problem.

[mario6714]

Ok, thanks

[mario6714]

One question (to avoid opening a new issue), I need to redownload everything again to use the default name, right?

For update

KMPDownloader.py -v -u -o -m -t 2 --UPDATE --RENAME

For download a new artist

KMPDownloader.py -v -u -o -m -b -t 2 -d D:\KMPDownloader

[Jeffchen54]

For the first question, you will need to use "KMPDownloader.py -v -u -o -m -t 2 --RENAME -d "directory where the work is"" for the works:

In order to change their names into the default format. So yes, you'll need to redownload them. If you don't want to go through the post one by one, you can use the --REUPDATE switch instead of the --UPDATE switch in "KMPDownloader.py -v -u -o -m -t 2 --UPDATE --RENAME"; however, this will download all the zip files again which will take up space and time.

For the second part, you don't need --RENAME when using update. --UPDATE downloads new works so it will not redownload any past works. The switch to download a new artist will work fine.

Let me know if you have any more questions about usage.

[mario6714]

Example:

KMPDownloader.py -v -u -o -m -t 2 --RENAME -d "D:\KMPDownloader\Kamuo" like this?

[Jeffchen54]

Yes, that should work.

Edit: set -d to KMPDownloader, not kamuo

[mario6714]

Not working (or i think is for something different):

I don't want me to ask the link, that is for re download all no?

i don't have the space for re download all, i need delete all for use the new format?

[mario6714]

Like this for re download?

KMPDownloader.py -v -u -o -m -t 2 --REUPDATE --RENAME

Or i need first delete all before use that command?

[Jeffchen54]

Some files cannot be renamed during redownloading, I suggest either downloading an artist or a work one at a time and removing what isn't renamed (this will save on internet bandwidth) or remove the existing files (keep the artist directory but it can be empty, this is the directory in KMP.db) and using the switches "KMPDownloader.py -v -u -o -m -t 2 --REUPDATE --RENAME". I recommend the first approach since this is early software and there are already stuff I need to work on but do what fits you the most.

[mario6714]

I am deleting all and only left the root folder (artist folders):

this is for re download all using the new format?

KMPDownloader.py -v -u -o -m -t 2 --REUPDATE --RENAME

or i need add another parameter for make better and avoid duplicates?

[mario6714]

and for update after that is ok like this?

KMPDownloader.py -v -u -o -m -t 2 --UPDATE

or i need add another parameter for make better and avoid duplicates?

[mario6714]

and for new artists like this is ok right?

KMPDownloader.py -v -u -o -m -b -t 2 -d D:\KMPDownloader

or i need add another parameter for make better and avoid duplicates?

[Jeffchen54]

I am deleting all and only left the root folder (artist folders):

this is for re download all using the new format?

KMPDownloader.py -v -u -o -m -t 2 --REUPDATE --RENAME

or i need add another parameter for make better and avoid duplicates?

The switches you gave should work fine.

[Jeffchen54]

and for new artists like this is ok right?

KMPDownloader.py -v -u -o -m -b -t 2 -d D:\KMPDownloader

or i need add another parameter for make better and avoid duplicates?

This is correct too.

[Jeffchen54]

and for update after that is ok like this?

KMPDownloader.py -v -u -o -m -t 2 --UPDATE

or i need add another parameter for make better and avoid duplicates?

This is correct.

[Jeffchen54]

I am updating thr wiki and will release a guide to using it alongside a hot fix within a week if there's any new issues that crop up besides the ones I will be addressing. It's fine to use right now however you can wait until it's more stable and all documentation is ready if you want to make sure it will work perfectly.

[mario6714]

Ok.thanks

[mario6714]

i am using this release:

"https://github.com/Jeffchen54/KMP-Kemono-Downloader/releases/tag/v0.6.2.3_reupload-pre"

is the most recent right?

[Jeffchen54]

Yes

[mario6714]

two things

Still downloading since yesterday and the only think downloading is comments (txt) files, nothing more

and i see re update or update don't respect the other parameters, i don't want comments or txt files and still is the first think your program downloads

[mario6714]

1 day already running and only txt files (comments)

[mario6714]

In this moment is running like this:

KMPDownloader.py -v -u -o -m -t 2 --UPDATE

i delete the data in the field "latest":

[Jeffchen54]

I'll take a look at issue where comments and stuff are being downloaded when not specified when I have time. Reupdate may take a substantial amount of time if you have a lot of works in your database. If the bar shows up, it means images and other items are being downloaded. As long as it says it is scanning works, it is still fine, there isnt a way to make it any faster because of rate limiting. The program downloads post content and stuff when scanning the works.

[mario6714]

i think your program work in the next way, scan all the artists, compare vs the last url for that artist and download in the end right?

is not better scan 1 artist, compare vs the last url, download and save in the DB and go to the next artist and repeat?

[mario6714]

i think i know why in that way but what happen if you have 100 + and the internet go offline or the site go offline or other problem?

Is start again i think

it's just a thought

[mario6714]

a way to avoid this "The program downloads post content and stuff when scanning the works."?

[mario6714]

i use "-o -m" for avoid that i think,

here the parameters i use:

-v -u -o -m -t 2 --UPDATE

[mario6714]

a way to avoid this "The program downloads post content and stuff when scanning the works."?

This is new? i cant remember have that files before use the last release and tried to download all again

[Jeffchen54]

If you open up KMPDownloader.py. Find the line "if not self.exclcomments and "patreon" in url or "fanbox" in url:" and replace it with "if not self.exclcomments and ("patreon" in url or "fanbox" in url):". It should fix the issue.

[mario6714]

for the moment is ok, because for do that i need start again, already running for 1 day "and keep going" jajajaja

[mario6714]

Thanks

[Jeffchen54]

i think i know why in that way but what happen if you have 100 + and the internet go offline or the site go offline or other problem?

Is start again i think

it's just a thought

You can go through one artist at a time. Instead of doing --REUPDATE, use your normal switches with --RENAME and -d to where the artist directory is located in and paste the artist URL. I have some protections against connection losses but I'm not sure if it will work on all cases,. It's good enough to where if you are starting to download the work (where the bar is showing) and put your pc to sleep, it will continue from where you left off.

[mario6714]

The problem in that is the time, i already deleted all for download all

[mario6714]

Is still scanning

[Jeffchen54]

I understand, --REUPDATE should redownload everything you removed, I've tested it with something that took a week to complete and it took a few days to finish scanning, as long as the console doesn't freeze, you'll be fine.

[Jeffchen54]

I'm still working on a patch for bugs, progress has been made but Kemono went down for most of the day which slowed things down quite a bit. Let me know if there's any other issues you run into.

[mario6714]

in the end i opened the .db file, delete the data in latest, saved and used:

KMPDownloader.py -v -u -o -m -t 2 --UPDATE

and thanks, just take a lot of time to download everything. can you do something for the case when for some reason can't download a file, just stop the program (after retrying like 100 times) and save the progress to the database?

[Jeffchen54]

You can use the -r <# of retries> switch. There's actually a bug that has been fixed but not released where the number of retries are infinite so you'll have the use -r 100 to get what you want. It will write the failure to log if it has to retry that many times and move on to the next file.

[mario6714]

You can use the -r <# of retries> switch. There's actually a bug that has been fixed but not released where the number of retries are infinite so you'll have the use -r 100 to get what you want. It will write the failure to log if it has to retry that many times and move on to the next file.

I think is better to put as default , if that happen skip that file or save the progress and stop the program

[mario6714]

These are just ideas, thanks for the program, it really helps

[Jeffchen54]

Yeah, I've implemented it already, just testing what I have currently before I release a hotfix to fix the few critical issues and --RENAME switch before working on general stability for the rest of the program. I'll have it at 10 retries at default since a file probably is down if refreshing the connection 10 times does nothing.

[mario6714]

Thanks for your program and your time

[mario6714]

One question, still want this "issue" open?

[Jeffchen54]

Yeah, I'll close it myself when a release is made fixing the issues here.