Bionus / imgbrd-grabber

Very customizable imageboard/booru downloader with powerful filenaming features.
https://www.bionus.org/imgbrd-grabber/
Apache License 2.0
2.55k stars 216 forks source link

Monitor "Last state" status and "Last success" date not correct #2918

Closed VeggieJay closed 1 year ago

VeggieJay commented 1 year ago

Bug description

I have two monitors, a and b with different issues

a) Mixed "last state" is not updating "last success" where any sources have "[Warning] No results for monitor 'x'" on site 'y'". So the monitor runs from an old date even though it has technically been successful. My example for this is "wildeerstudio"

b) Sometimes there is no error and no warning, but monitor still has mixed "last state" and does not update "last success". Monitor runs successfully otherwise. My example for this is "xuuikie_ashe"

Steps to reproduce

  1. I have 5 primary sources: sankaku, danbooru, gelbooru, rule34.xxx, yande.re
  2. Create a monitor for example tag that exists on some of the sources but not others. Tag "wildeerstudio" exists on sankaku and rule34.xxx
  3. Create a monitor for example tag that exists on all 5 sources. Tag "xuuikie_ashe" exists on all 5 sources.
  4. See whether monitor "last state" and "last success" are updating

Expected behavior

a) Maybe "last state" and "last success" should succeed with warnings but not with errors? I am not sure of a way to resolve this without accidentally skipping files, however it is very common when doing searches across multiple sources for tags to exist on some sites and not others.

b) Last state/last success should be updating normally for this monitor since it is running without any other issues

Context main.log (redacted my api keys)

settings.txt (change extension for settings to .ini, couldn't upload to github without changing file extension)

monitors.txt (change extension for monitors to .json, same issue)

Screenshots

Capture

System information

Bionus commented 1 year ago

a) Mixed "last state" is not updating "last success" where any sources have "[Warning] No results for monitor 'x'" on site 'y'". So the monitor runs from an old date even though it has technically been successful. My example for this is "wildeerstudio"

The thing is "No results" is different from "No new results", and is not considered as a success (although there's also a bug where "no new results" isn't considered as a success either).

The fact you're getting "no results" means your search is returning nothing at all, before or after the "last success date". Usually, that means the tag could have been renamed, or a bug in the source (like Danbooru since last week). That's why it's not considered as a success, otherwise a temporary issue in the source that would not directly fail but just return nothing would increase the "last success" and cause you to miss the images uploaded during that time.

In your case as you said, it's simply that the tag does not exist on that source at all. Why monitor it and cause additional traffic then? I guess it could have a different setting for this, but as you said that might cause missed images.

b) Sometimes there is no error and no warning, but monitor still has mixed "last state" and does not update "last success". Monitor runs successfully otherwise. My example for this is "xuuikie_ashe"

Yes that's a bug. Just pushed a fix 👍

VeggieJay commented 1 year ago

Do you think we can have a setting where if there are no results in the search, when you press "Monitor" then it will only add the sources that have results? That should fix the issue entirely moving forward except for those cases that you mentioned where the tag changes later

IIRC right now the monitor automatically adds all sources whether there are results or not

Bionus commented 1 year ago

Yes that makes sense. Just added such setting, and by default empty sources won't be monitored anymore.