Bionus / imgbrd-grabber

Very customizable imageboard/booru downloader with powerful filenaming features.
https://www.bionus.org/imgbrd-grabber/
Apache License 2.0
2.55k stars 216 forks source link

[Possible Bug(s)] Searches Failing to Load and Batch Downloads Not Completing #2157

Closed plashomb closed 1 year ago

plashomb commented 3 years ago

Running Grabber v7.3.2 on Windows 10 on PC

Issue 1: Searches Failing to Load (Possible bug?)

Issue Description:

Occasionally, when I hit the "ok" button to perform a search for a booru and hit "ok" again before it's able to finish loading everything, the results completely disappear for the booru. It's not that it shows "booru_name: no search results"; it just doesn't display anything.

Image of Issue:

Here's an example of what it looks like after hitting "ok" to search using safebooru and then hitting "ok" again before it's able to finish loading:

c9ezK1

Replicating Issue:

I can sometimes replicate it by clicking "ok" and then clicking "ok" again before it finishes loading, but sometimes I have to click the "ok" button several times before it will trigger the issue. However, I think that in the cases where I have to click more than twice, it's because it's loading too quickly.

Troubleshooting:

So far I've tried using different builds of Grabber and the same issue occurs. The only way that I can "fix" the error is by restarting the application, entirely.

Log File for Example of Issue:

[21:17:37.742][Info] Loading results...
[21:17:37.744][Info] [safebooru.org][Html] Loading page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:37.744][Info] [safebooru.org][Xml] Loading page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:38.779][Info] [safebooru.org][Xml] Receiving page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:38.850][Info] [safebooru.org][Xml] Parsed page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!: 100 images (100), 0 tags (676), 3702 total (3702), 38 pages (38)
[21:17:39.286][Info] [safebooru.org][Html] Receiving page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:39.307][Info] [safebooru.org][Html] Parsed page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0: 40 images (40), 54 tags (54), -1 total (3696), 88 pages (88)
[21:17:46.524][Info] Loading results...
[21:17:46.533][Info] [safebooru.org][Html] Loading page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:46.533][Info] [safebooru.org][Xml] Loading page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:47.295][Info] [safebooru.org][Html] Receiving page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:47.316][Info] [safebooru.org][Html] Parsed page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0: 40 images (40), 54 tags (54), -1 total (3696), 88 pages (88)
[21:17:47.807][Info] Loading results...
[21:17:47.814][Info] [safebooru.org][Xml] Receiving page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:47.815][Warning] [Qt][default] QIODevice::read (QNetworkReplyHttpImpl): device not open
[21:17:47.815][Warning] [safebooru.org] Loading using Xml failed. Retry using Html.
[21:17:47.817][Info] [safebooru.org][Html] Loading page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:47.818][Info] [safebooru.org][Html] Loading page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:47.818][Info] [safebooru.org][Xml] Loading page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:48.557][Info] [safebooru.org][Html] Receiving page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:48.578][Info] [safebooru.org][Html] Parsed page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0: 40 images (40), 54 tags (54), -1 total (3696), 88 pages (88)
[21:17:49.997][Info] [safebooru.org][Html] Receiving page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:50.021][Info] [safebooru.org][Html] Parsed page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0: 40 images (40), 54 tags (54), -1 total (3696), 88 pages (88)
[21:17:50.491][Info] [safebooru.org][Xml] Receiving page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:50.559][Info] [safebooru.org][Xml] Parsed page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!: 100 images (100), 0 tags (676), 3702 total (3702), 38 pages (38)
[21:17:51.521][Info] Loading results...
[21:17:51.524][Info] [safebooru.org][Html] Loading page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:51.525][Info] [safebooru.org][Xml] Loading page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:53.046][Info] [safebooru.org][Html] Receiving page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:53.068][Info] [safebooru.org][Html] Parsed page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0: 40 images (40), 54 tags (54), -1 total (3696), 88 pages (88)
[21:17:53.251][Info] [safebooru.org][Xml] Receiving page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!
[21:17:53.319][Info] [safebooru.org][Xml] Parsed page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!: 100 images (100), 0 tags (676), 3702 total (3702), 38 pages (38)
[21:17:53.416][Info] Loading results...
[21:17:53.426][Info] [safebooru.org][Html] Loading page https://safebooru.org/index.php?page=post&s=list&tags=kono_subarashii_sekai_ni_shukufuku_wo!&pid=0
[21:17:53.426][Info] [safebooru.org][Xml] Loading page https://safebooru.org/index.php?page=dapi&s=post&q=index&limit=100&pid=0&tags=kono_subarashii_sekai_ni_shukufuku_wo!

Issue 2: Batch Downloads Not Completing (Maximum Download limit?)

Issue Description:

Batch downloads are occasionally failing to complete and stopping about half-way through downloading.

Image of Issue:

It looks sort of like this when it happens (obviously it's not paused when it does it, though):

kYJI3O

Replicating Issue:

I've been unable to determine precisely what is triggering it in order to replicate the issue, but it generally happens after a lot of batch downloads. As a very, very rough estimate, the issue occurs after downloading perhaps somewhere around ~2000-3000+ images total.

Troubleshooting:

I've tried closing the batch download window and starting the batch download again, but it doesn't download anything. I've also tried multiple versions of Grabber builds and the same issue persists. The only way that I've been able to fix it is by closing and restarting the entire application.

Issue 3: Images Failing to Load (Maximum Download limit?)

Issue Description:

Images in the image viewer are occasionally failing to load completely. They remain blurry as if they're trying to load, but fail to do so.

Image of Issue:

Here's an example of what the issue looks like when it happens:

vNMDoR

Replicating Issue:

So far, I've been unable to determine what triggers it in order to replicate the issue, but it generally happens after viewing a lot of images or downloading a lot of images.

Troubleshooting:

The only way I can fix it is by closing and restarting the application, entirely. I've tried multiple versions of Grabber and the issue persists. Is there some sort of maximum download limit option that I'm missing that's causing both issue 2 and 3?

Thanks a lot for any assistance!

KeyboardCoomer commented 3 years ago

I've definitely experienced similar issues regarding having too many groups cued up causing a soft-crash. When it occurs, the only thing that seems to alleviate the issue is to close and boot-up the program again - which clears the saved groups.

Just as a clarification, when I'm referring to the soft crash, I mean the program seems unable to access the internet. I've double checked with task manager, and there was no network activity, up or down.

Also, in the case where the image no longer loads when previewing it, I would double-check to see if the image exists on the website you're pulling from. I've had some experiences where the image grabber is able to find images that no-longer exist on the site.

Also, I think part of what "pauses" the software is when you have the program in the background. I've had to literally click on the program to get it to resume downloading.

plashomb commented 3 years ago

Thanks for the response!

Closing it and rebooting it is the only way I've been able to fix it.

Yup, they all exist. When it no longer loads, I always try clicking on a lot of other images to check and it still doesn't manage to load.

I've tried fiddling with clicking on the program to get it to resume downloading when it "pauses", but it doesn't seem to get it to fire back up, again, unfortunately.

KeyboardCoomer commented 3 years ago

That's unfortunate to hear about the inability to resume downloads. I've been trying to ensure I'm not downloading more than 2000 or 3000 items at a time, or space them out if I can. I wonder if it's possible that this program takes a lot more computing or network power than it leads on.

plashomb commented 3 years ago

That's entirely possible. It just gets tedious having to restart the program every few batch downloads or so.

KeyboardCoomer commented 3 years ago

That's unfortunate to hear about the inability to resume downloads. I've been trying to ensure I'm not downloading more than 2000 or 3000 items at a time, or space them out if I can. I wonder if it's possible that this program takes a lot more computing or network power than it leads on.

KeyboardCoomer commented 3 years ago

Encountered the bug again, not exactly sure why. But some photos as proof for it essentially being unable to access the internet after something triggers. image image As you can see, there are other programs running and accessing the internet without an issue, but the grabber just suddenly disconnects itself.

KeyboardCoomer commented 3 years ago

A quick fix I actually found was going into settings and allowing the program to restore previous session and go to my AppData/local files for grabber and click on the "restore.igl" file. Surprisingly, it keeps all tabs open, all downloads queued, and seems to fix the issue with connectivity.

The restore last session on start thing may not be necessary. But I think until the issue is pinned down, just refer to the restore.igl

stale[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

If this issue is about a bug that still happens in the latest version, or a suggestion that is still relevant, feel free to comment on it and the maintainers will have another look, they might have missed it!

Thank you!