Normal notification not sent after alert

[unka65]

If the threshold trigger process is being overhauled this is irrelevant. Otherwise, hopefully this is helpful. I noticed similar in the past but did not determine the cause then as I was less familiar with Thold. This is with thold develop version as of early September 2023.

A threshold has the following settings:

The alert notification occurred at 2:10 as expected but no normal notification occurred later.

Debug level logging was not enabled to confirm but from the graph the threshold sequence seemed to be:

• Alert notification
• Warning level. If I interpreted the code correctly, the alert count was cleared at thold_function.php line 2533
• Alert level (not required for scenario)
• Normal level before warning notification trigger count. Warning and alert counts were below trigger counts so no notification of threshold normal status.

The other level (warning or alert) count is cleared when the new level occurs, before a notification count is reached and so does not indicate a previous notification when back to normal. As a workaround, I changed to clear the other level counter when the new notification occurs. I thought about various scenarios of level changes (normal, warning, alert) including where could bounce between warning and alert but never notify. I do not know if that actually occurs anywhere and is important to go into detail and to justify any changes.

[TheWitness]

@xmacan, can you review this one?

[TheWitness]

This may have been fixed for Non-Template tholds yesterday.

[TheWitness]

@xmacan, can you comment?

[TheWitness]

The fix here is to refresh the develop version, then set your version back to 1.5 in the plugin_config table and then re-upgrade. I think it'll start working again afterwards.

[xmacan]

Testing with my db fix but still not working, tholds are still triggered:

[unka65]

I am the rookie outsider but this is in my reported issue so ... I noted that my values have units (no matter what I change) unlike in the values in the screenshot. I reviewed the code and it seems straightforward. It has probably already been checked but I wonder what values are listed in either the debug log or preferably via the table query.

SELECT id, name_cache, thold_hi, thold_warning_hi, thold_low, thold_warning_low, lastread, show_units, units_suffix, expression FROM thold_data WHERE id in (7781, 7789)\G

[TheWitness]

So, this is still open then guys? Can someone refresh me on the current status?

[xmacan]

I can see this issue with actual develop branch. @unka65 could you test the same thing with actual develop branch?

[unka65]

I have not had unexpected thresholds reached and so I do not know how to recreate it. Does the thold debug level logging have any clues? What are the values in the threshold?

As for the notification process, I reworked the high/low notifications several weeks ago. The notifications have been working as I would expect, fixing the following.

type high/low
Alert count 2 
Warn count  3

Missing notify to Normal (this initial issue)
# level count, Status (level) n-normal w-warning a-alert (upper of each is notify)
        ^... ... 1 ... ... 2
Alert #    120 1
Warn #    1  1
Normal  nn     n
Status  nnwaAwan
Notify      A  !< no Normal notify triggered
Expect      A  N

Unexpected notify after maintenance mode
Time    ^... ... 1 ... ... 2
Maint    mmm
Alert #  12340
Warn #  
Normal  n    n
Status  naaaaN
Notify       N!
Expect

No Alert to Warning - Include pre notify Alert count in Warning count/notify.
        ^... ... 1
Alert #  120
Warn #     123
Normal  n     n
Status  naAwwWn
Notify    A  WN
Expect    A
          A>W N

I do not remember considering the shown status of the threshold in the threshold list and probably one reason I did not mention the changes.

On my system, I will merge my changes with the latest develop version and monitor it.

An unrelated, error fix in my version is the following.

ERROR PHP WARNING in Plugin 'thold': Undefined array key "thold_id" in file: 
/var/www/html/cacti/plugins/thold/thold_functions.php on line: ~3528
function get_email_subject($phase, $trigger, $lastread, $ra, $breach_up, &$thold_data) {
...
case '1': // Baseline
            if ($breach_up) {
                $value = $thold_data['thold_id']; < to 'thold_hi'
            } else {
                $value = $thold_data['thold_low'];
            }

[TheWitness]

This is fixed now. I have a few more comments on your pull request @unka65. Thanks for your patience. Note, I've bumped the minimum Cacti version to 1.2.27. Since you are on Thold 1.8, you will have to run this SQL command and then re-upgrade too:

UPDATE plugin_config SET version='1.7' WHERE directory='thold';