Bionus / imgbrd-grabber

Very customizable imageboard/booru downloader with powerful filenaming features.
https://www.bionus.org/imgbrd-grabber/
Apache License 2.0
2.52k stars 216 forks source link

Gelbooru: 0 Downloaded, EVERYTHING exists #443

Closed wtbguest closed 8 years ago

wtbguest commented 8 years ago

This only started happening today, I've used the program for over a year with no issues, so I can't tell if this is just a random connectivity issue or something else.

Example - When looking through the tag tohsaka_rin in the browser within the program it shows: -Page 1 of 171 (20 of 3414) -Click "Get All" and it adds the tag to downloads, sets the images per page to 200 and the max image limit to 3414. -When I try to batch download it speeds through the list almost instantly, greenlighting all results, and says: 1 file downloaded successfully. 0 ignored files. 755 files already existing. 0 files not found on server -When I check the folder the only image in it is a blurry jpg thumbnail of a random image from the tag. Sometimes I've gotten 2 where the second is a gif, but it's still a static thumbnail(same size you'd expect on the actual site when visiting). They never even have filenames, just entirely blank in the text field.

My filename setting is %md5%.%ext% as it's always been.

Worth noting too that whenever I try to combine two tags(i.e. tohsaka_rin swimsuit) the pictures show up in the programs viewer, but it says "Page 1 of ? (20 of ?)". Same result when downloading('2 downloaded, rest exist', no filename thumbnail). This worked until today as well. Tried several tags, tag combinations, filename settings, folder locations and image per page settings(1000, 200, 20). Any idea whats going on? I didn't even touch the program between my last use a few days ago and the first borked attempt today, so I didn't change anything before this happened. Did Gelbooru catch on to our shenanigans?

I'm on Windows 8.1 64bit using 4.1.8. I only updated the program today. Been using 3.1.2 or something, but had this problem start which is why I updated in the first place, thinking it might fix it.

Bionus commented 8 years ago

Gelbooru blocks API access to non-logged in users.

You should either login using the credentials tab of the source settings window (you can open it by clicking "Sources" then clicking the settings button near Gelbooru. If you don't have an account on Gelbooru and don't want to create one, the only valid "Images per page" value is 42.

Does this fix the problem? I'll try to make it more clear in future updates.

wtbguest commented 8 years ago

Bionus, Unfortunately that doesn't appear to help. I don't think I ever had an account on Gelbooru before. But I made one to try the solution out, and all I get on entering the information on both the Credentials and Login tabs is "Error" after hitting the test button. It save the information in the /Documents/Grabber/Sites/Gelbooru/settings.config file correctly. Being logged in or out of the site in a web browser doesn't seem to change anything.

Setting the images per page limit to 42 didn't help either; I still got my filenameless thumbnail and a report that everything it tried to download(only a fraction of the total image count) already existed. These aren't ones I've downloaded before, even through other tags, so I don't imagine it's a "skip all downloaded" problem, is it?

If Gelbooru with Grabber still works normally on your or someone elses set up, I think I'll just resign to the fact that somehow, somewhere, my computer or internet just changed slightly and ruined it for me. I don't see why this would change over night like this, or why it would suddenly think everything exists and if not just download a thumbnail or two without running it through name generation.

imgbrd commented 8 years ago

i've got the same problem - i'll try to login it but it'll be unable to test nor will it actually solve the problem

EDIT: alright, so i'm now able to login, but logging in doesnt actually solve anything beyond browsing. downlaoding still doesnt download the correct amount of images, and on soem tags just defaults to 42 anyways. sometimes even when i log in it won't even work and it'll jsut be stuck loading 42 images.

wtbguest commented 8 years ago

imgbrd, Yup, experiencing the same things now. Logging in works but doesn't fix the issue. Sounds like Gelbooru just changed things up or something though I can't really think of how. Paheal and others still seem to work fine, anyway. Go figure it's my primary source that throws a wrench into things.

Bionus commented 8 years ago

I don't think I ever had an account on Gelbooru before. But I made one to try the solution out, and all I get on entering the information on both the Credentials and Login tabs is "Error" after hitting the test button.

Which would mean the entered credentials are incorrect. Note that you don't have to hash the password. The 4.1.9 fixes many issues with login testing so you might want to try it out when I release it this week-end.

Being logged in or out of the site in a web browser doesn't seem to change anything.

Yes, that's normal.

Setting the images per page limit to 42 didn't help either; I still got my filenameless thumbnail and a report that everything it tried to download(only a fraction of the total image count) already existed. These aren't ones I've downloaded before, even through other tags, so I don't imagine it's a "skip all downloaded" problem, is it?

No, that means the program failed to fetch or parse the data received from the server and could not determine the %md5% token value (so they all had the same name in the end). Which is quite weird since the 4.1.8 fixes the regex source, and even weirder if you can browse the images correctly before.

If Gelbooru with Grabber still works normally on your or someone elses set up, I think I'll just resign to the fact that somehow, somewhere, my computer or internet just changed slightly and ruined it for me. I don't see why this would change over night like this, or why it would suddenly think everything exists and if not just download a thumbnail or two without running it through name generation.

The "overnight changes" are due to Gelbooru currently changing and adding multiple checks on both their API and normal browsing pages to prevent users from abusing it, which kinda broke the site around a month ago.

Bionus commented 8 years ago

EDIT: alright, so i'm now able to login, but logging in doesnt actually solve anything beyond browsing. downlaoding still doesnt download the correct amount of images, and on soem tags just defaults to 42 anyways. sometimes even when i log in it won't even work and it'll jsut be stuck loading 42 images.

In that case I guess 4.1.9 should fix this problem. Seems like 4.1.8 starts to download pages before even logging in during batch downloads, so it's kinda problematic. Works fine for now on my dev version, so I'll release that during the week-end.

Bionus commented 8 years ago

The 4.1.9 is out, does this fix the issue ? Make sure to login first if you don't want to be limited to 42 images per page. I hope it should end all problems with Gelbooru.

wtbguest commented 8 years ago

Yup, works perfectly now. Thank you, Glorious Leader.