dgtlmoon / changedetection.io

The best and simplest free open source web page change detection, website watcher, restock monitor and notification service. Restock Monitor, change detection. Designed for simplicity - Simply monitor which websites had a text change for free. Free Open source web page change detection, Website defacement monitoring, Price change notification
https://changedetection.io
Apache License 2.0
16.84k stars 940 forks source link

Empty diff triggers change detection #2548

Open Noki opened 1 month ago

Noki commented 1 month ago

Describe the bug

I sometimes get change detection notifications even though the diff is empty. My notification body setting looks like this:

{{watch_title}} / {{watch_url}} had a change.<br><br>

see diff: {{diff_url}}<br>
---diff---<br>
{{diff}}<br>
---diff---<br>

Whenever it happens the notification looks like this:

image

[...] had a change.<br><br>

see diff: http://diskstation:3003/diff/040867f4-9a48-4a24-b4e1-9f6b2b84b626<br>
---diff---<br>
<br>
---diff---<br>

In addition I can't download the latest HTML snapshot:

image

Version

v0.46.02

To Reproduce

Can't reproduce. Can't share the check, as it contains sensitive information. Best guess: An internal error is not handled properly. It might have to do with the "Remove elements" and "Block change-detection while text matches" features, which I both use for the check. Maybe a blocked change detection still stores data and the following run is incorrectly compared against it.

I found the following in the docker logfile:

image

It clearly states that the hash is different.

image

Expected behavior

Never send out notifications in case of an empty diff. Do not detect a change in this case.

petersg83 commented 1 month ago

Do you have any text to ignore in the diff? I have the same issue and my intuition is that the text to ignore triggered the notification whereas it shouldn't. And it causes an empty notification if only the text to ignore changed.

Noki commented 1 month ago

The ignore text is indeed the diff. I'm checking a service which produces unstable results. Sometimes it responds with a result list, sometimes it can't access it's database and responds with a error message e.g. "No results available". That error message is exactly the text I ignore as I am only interested in real result changes.

SansGuidon commented 3 weeks ago

It's quite annoying, I've hundred of watches and have configured texts to be ignored globally and per watch, I would love an option or more control on what kind of diff triggers a change.

Note that I'm following changes on my changedetection instance through RSS feed.

dgtlmoon commented 3 weeks ago

@SansGuidon

It's quite annoying, I've hundred of watches and have configured texts to be ignored globally and per watch, I would love an option or more control on what kind of diff triggers a change.

Can you be any more specific here? are you saying the global ignore doesnt work or? what is the problem you are seeing exactly?

dgtlmoon commented 3 weeks ago

@Noki

The ignore text is indeed the diff.

Yes that is 100% correct, it SHOULD be in the diff, "ignore text" only ignores the text when detecting the difference, ofcourse, the application should show you the whole difference

SansGuidon commented 3 weeks ago

@dgtlmoon

@SansGuidon

It's quite annoying, I've hundred of watches and have configured texts to be ignored globally and per watch, I would love an option or more control on what kind of diff triggers a change.

Can you be any more specific here? are you saying the global ignore doesnt work or? what is the problem you are seeing exactly?

I get notified about changes both through RSS feed and through Changedetection UI. However the diffs shows nothing has changed, because the changes are indeed ignored globally or per watch.

In the RSS feed, it will result as empty content (just a link). In Changedetection UI, nothing interesting is shown, of course I can click "show current snapshot" then I see the texts that were ignored.

Soe the issue is indeed that despite the ignore text feature works, we are yet notified about changes, while there is none to be shown.

Noki commented 3 weeks ago

@Noki

The ignore text is indeed the diff.

Yes that is 100% correct, it SHOULD be in the diff, "ignore text" only ignores the text when detecting the difference, ofcourse, the application should show you the whole difference

Ok. So how do I avoid triggering a change in this case? Any suggestions?

dgtlmoon commented 3 weeks ago

@Noki

Ok. So how do I avoid triggering a change in this case? Any suggestions?

Add the text that might change - which you want to ignore in the change detection - to the various ignore text boxes in the [Edit] tab of the watch, use the "preview" to Visualise what would be ignored (it will highly ignored lines in grey)

Did you find that part in the UI? Espicially the last part?

Noki commented 3 weeks ago

@dgtlmoon

I don't want to ignore text. I want to block the change detection completely. I am using this one but it does not work properly:

image

The markup of the HTML is quite simple so a Text comparison should be fine.

image

I added both texts exactly as is and they should match and stop the change detection.

Note: The last time I had the error I described was at Agust 9th, so I can't really tell if it is fixed now, or if the website I am monitoring is just more stable these days. Will get back to you on this if it happens again.

dgtlmoon commented 3 weeks ago

@Noki click [Diff] from the main list, then [Show current snapshot] in the top left corner, you should see that text (I assume in your blacked-out box) should still show in the diff because its part of the snapshot, can you confirm it? also hit [recheck]