search

Nandaka / PixivUtil2

Download images from Pixiv and more!
http://nandaka.devnull.zone/
BSD 2-Clause "Simplified" License
2.37k stars 257 forks source link

Question: Confusion about image download delay in 20200101 #612

Closed bnick23 closed 4 years ago

bnick23 commented 4 years ago

Hello! Was running the latest release when I noticed the downloaddelay (which is set to 10) was happening not just for new downloads like it used to, but when checking the already downloaded ones as well, adding a lot of extra time. Is this a new intended change? I apologize if its mentioned somewere and I missed it.

Old previous 20191221 version downloaddelay = 10 example:

PixivDownloader20191221 old

New 20200101 version downloaddelay = 10 example:

PixivDownloader20200101 new

Nandaka commented 4 years ago

most likely due to #609, might be duplicated with #613. replace the library.zip from https://github.com/Nandaka/PixivUtil2/releases/tag/v20200102-beta1

ghost commented 4 years ago

most likely due to #609, might be duplicated with #613. replace the library.zip from https://github.com/Nandaka/PixivUtil2/releases/tag/v20200102-beta1

Tested with v20200102-beta1, issue still exists with https://github.com/Nandaka/PixivUtil2/issues/613#issuecomment-570106587

Nandaka commented 4 years ago

try library.zip

ghost commented 4 years ago

try library.zip

Tried. But the issue still exists. Using v20191221's DB and config, replaced v20200101‘s library.zip with the new one, doing "8. Download new illust from bookmarked members (/bookmark_new_illust.php)".

Nandaka commented 4 years ago

updated library.zip

ghost commented 4 years ago

updated library.zip

Tested again, issue still exists.

I use v20191221, not have this issue.

Nandaka commented 4 years ago

updated again library.zip

ghost commented 4 years ago

updated again library.zip

Tested again again with library.zip (SHA1: 5DDEE56DBEBCAED0406C24415DF12FB94F0436E8), issue still exists using "8. Download new illust from bookmarked members (/bookmark_new_illust.php)".

Oh no...

Nandaka commented 4 years ago

do you enable alwaysCheckFileSize or overwrite?

ghost commented 4 years ago

do you enable alwaysCheckFileSize or overwrite?

Both of them are set to False.

I'll move my downloaded files to another folder several days after downloaded. v20200101 with patch will only check files in the download folder, if file already downloaded but not in the folder, even if the history exists in DB file, it will still do the downloading. v20191221 will check download history in DB first, if it is in DB history, it will skip the file, even if the file not exists. I think that's the reason cause this DB issue.

Nandaka commented 4 years ago

updated library.zip

ghost commented 4 years ago

updated library.zip

Tested again again again with library.zip(SHA1: 0DF37F02BCB609E73D45DC2F7381F9DB10295E03) and v20200103-beta2, issue still exists.

After v20200101, it seems not check DB history first. I'll roll back to v20191221 without this issue.

Nandaka commented 4 years ago

https://github.com/Nandaka/PixivUtil2/issues/613#issuecomment-570280699

Ensure alwayscheckfilesize and overwrite is set to False in config.ini, else it will have some delay enabled as it actually retrieve the image information from pixiv server.

ghost commented 4 years ago

updated library.zip

Tested again again again with library.zip(SHA1: 0DF37F02BCB609E73D45DC2F7381F9DB10295E03) and v20200103-beta2, issue still exists.

After v20200101, it seems not check DB history first. I'll roll back to v20191221 without this issue.

#613 (comment)

Ensure alwayscheckfilesize and overwrite is set to False in config.ini, else it will have some delay enabled as it actually retrieve the image information from pixiv server.

Running mode 8. Alwayscheckfilesize and overwrite are both set to False in config.ini, issue still exists when files already downloaded but not there.

Nandaka commented 4 years ago

Weird, I'm running from the source and it is working as intended..

Try the full release here? https://github.com/Nandaka/PixivUtil2/releases/tag/v20200103-beta2

ghost commented 4 years ago

Weird, I'm running from the source and it is working as intended..

Try the full release here? https://github.com/Nandaka/PixivUtil2/releases/tag/v20200103-beta2

Hummmm... Full release works for me. Thank you so much.

Nandaka commented 4 years ago

got it, look like I cannot release just the library.zip anymore...

ghost commented 4 years ago

got it, look like I cannot release just the library.zip anymore...

Replaced library.zip step by step, so what went wrong... Orz

bnick23 commented 4 years ago

Been running the latest release for a bit, and it seems to be working again. Thank you for your hard work!