catgirlfighter / nekopaw

galleries downloader
45 stars 15 forks source link

Images from pixiv download as jpg instead of png #340

Closed nawj closed 7 years ago

nawj commented 8 years ago

I found that when trying to edit a picture nekopaw downloaded that the image was a png and that nekopaw resaved it as a png meaning I have no idea whether these pictures were originally pngs without going through the rigamarole. Am I missing a setting where it's forcing this?

catgirlfighter commented 8 years ago

Can you give me an example of such image?

nawj commented 8 years ago

Warning, pretty much all of the linked images are NSFW

(https://my.mixtape.moe/ljsdzd.png image as downloaded with nekopaw) edit: mixtape automatically changed it to a .png after upload, so did imgur, so here is just an ss of files in the folder I downloaded to: https://my.mixtape.moe/bkinjt.png Everything downloaded in .jpg https://my.mixtape.moe/spxxif.png result of trying to open it in photoshop https://my.mixtape.moe/wequas.png opens fine in photoshop after changing the extension to .png

Also nekopaw ended up downloading just tons of duplicates https://my.mixtape.moe/iemobj.png but I scraped something that had a ton of images, I don't know if it was because it was such a large operation or if there were a bunch of duplicates in a bunch of different collections.

catgirlfighter commented 8 years ago

I required an URL on pixiv so I can test it, not actual pictures. I can't test it on already downloaded picture :)

catgirlfighter commented 8 years ago

Found a hint on last screenshot, I'll use it.

catgirlfighter commented 8 years ago

Something weird going on with your download. Used "Source: User works" without User ID checked (i.e. user id in tags). Found this: sample7 Downloaded an album 52820069. Got this: sample8 Windows Explorer, Firefox, Paint, GIMP say it's JPEG, all the same.

As well as it didn't give me duplicates. Did you download it more then once? If you can, please give me a full sequence of actions (mostly I need your set of presets for pixiv you used for search.

catgirlfighter commented 8 years ago

Oh, right, it's not an album that had a problem, I'm an idiot >< Basically, I need you to give me direct links to pictures, so I can actually test it on my side.

nawj commented 8 years ago

On 2016-09-05 05:01, CGFighter wrote:

Oh, right, it's not an album that had a problem, I'm an idiot >< Basically, I need you to give me direct links to picture, so I can actually test it on my side.

You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub [1], or mute the thread [2].

*

Links:

[1] https://github.com/catgirlfighter/nekopaw/issues/340#issuecomment-244730576 [2] https://github.com/notifications/unsubscribe-auth/AUshRW5-4j7KGQ_V7CQ63F9zU5vMfXOSks5qnASpgaJpZM4JzY1e Oh shit, sorry for taking so long to respond. I don't check this email often. Let me see if I can find a link.

Alright, I just used Nekopaw to download this image, NSFW http://www.pixiv.net/member_illust.php?mode=medium&illust_id=58921137 Image is a .png:http://i2.pixiv.net/img-original/img/2016/09/11/05/12/50/58921137_p0.png Caps of me downloading it with Nekopaw: https://my.mixtape.moe/ubvpnb.png https://my.mixtape.moe/qrgbyd.png Result file: https://my.mixtape.moe/jmfhah.png

Again, sorry for taking so long to reply. After I made a clusterfuck of a response to you on your git and making an ass out of myself I was kind of embarrassed and I forgot about it.

catgirlfighter commented 8 years ago

It's OK, it's not like I'm fast to respond anyway. Confirmed it. It really uses jpg extension when it original extension is PNG, and pictures real extension is PNG.

catgirlfighter commented 8 years ago

Should work now.

JarieSuisen commented 8 years ago

Still having the problem after the update, with both single and gallery downloads. Tested with author ID: 14776527

catgirlfighter commented 8 years ago

Be more specific. Picture 58921137 downloaded as PNG correctly for me. Please give me actual pictures, verifying every single picture quite troublesome :/

JarieSuisen commented 8 years ago

That's awkward... since that very image is a JPG on my download. The program even identifies the original on Pixiv as being JPG. example

But when I try to open the file in PSPV, it says it's a bad JPG file, and if I change the extension to PNG, it's fine.

catgirlfighter commented 8 years ago

For pixiv grabber uses "uncertain" method, means it doesn't know beforehand, which extension picture gonna be. So if grabber finds a downloaded picture with the same name, despite different extension, grabber gonna think it was already downloaded. I.e. you should delete old files. I use "uncertain" method to avoid loading extra pages, which can weight as much as picture itself.

JarieSuisen commented 8 years ago

Ok, I see that working (hadn't thought about the extension check before downloading), but only for single images. Each gallery download is still having the problem, such as 56803295.

catgirlfighter commented 8 years ago

Addressing that issue:

Also nekopaw ended up downloading just tons of duplicates https://my.mixtape.moe/iemobj.png but I scraped something that had a ton of images, I don't know if it was because it was such a large operation or if there were a bunch of duplicates in a bunch of different collections.

I downloaded it once, tried to download it second time, but grabber ignored it. Does it mean you tried to download from a lot of different sources at the same time, and they crossed? If that so, you probably should add a duplicate filter, something like "post_url = post_url". Normal url=url doesn't apply because for pixiv it's impossible to know a right url beforehand (old pictures have a different format of URL).

catgirlfighter commented 8 years ago

Mangamode supposed to have original extensions =^= Guess they changed it afterwards. Fixed.

JarieSuisen commented 8 years ago

Whoo! Looks like it's working right!

I know it's off the topic, but what about the newer animations (うごいら)? It's still failing to download them. (ex: image 59183003)

catgirlfighter commented 8 years ago

Gonna look into it. As always donno, how much time it's gonna take.

catgirlfighter commented 7 years ago

I know it's off the topic, but what about the newer animations (うごいら)? It's still failing to download them.

I guess I fixed it (accidentally).