Not receiving some notifications

[jindroush]

Description

One of my users was complaining he's not getting notifications for one of the subscribed threads - it was verified, that forum does not send any email (ie. not going to spam). I was getting the notifications for the very same thread (so notifications work). After long testing it seems that if user has both BOARD and THREAD notifications on, no notification gets through.

Steps to reproduce

1. Set board notifications to emails & notifications (this board contains thread from point 2)
2. Set thread notifications to emails & notifications
3. Wait for any message to the thread
4. No notification gets through.
5. If board notifications are turned off and new message is posted, the notification appears.

Environment (complete as necessary)

• Version/Git revision: 2.1.3

Additional information/references

I wasn't able to find any other explanation for the behavior described and I also wasn't able to find any other settings which explain it.

[nc3man]

I just confirmed this issue for SMF 2.1.3. If I have an open TOPIC set to receive emails and alerts and the corresponding BOARD is also set for emails and alerts, I get neither an email nor an alert on that TOPIC. If I remove the watch on the BOARD, then updates to the TOPIC correctly receive both an email and an alert.

[nc3man]

If a quick patch is available, rather than waiting for a full release, that would be appreciated. Most of our users who take the time to set a watch on a TOPIC also have watches set on all the BOARDs they frequent. This behavior is confusing people who want to follow an active topic.

[nc3man]

@sbulen Is there an estimate of when the 2.1.4 milestone might be released? If unknown or too far in the future, this bug is big enough for our users that I would be happy to install a patch (I'm pretty efficient with code) or should I consider reverting to a previous version?

[Sesquipedalian]

Is there an estimate of when the 2.1.4 milestone might be released?

I'm sure the answer, as always, will be "When it's ready." 😉

As far as I can see, no fix has been submitted yet, so there is no patch you could apply yet.

[sbulen]

I cannot reproduce this. If I set up notifications for a board (email + alert), and also setup notifications for a topic within that board (email + alert), I get the email & the alert after new posts to the topic.

I tested twice, both under 2.1.3 & under current GitHub.

I have two suspicions:

• There is a BIG distinction between "Immediate" and "Immediate but only for the first unread message". If the setting is set to the latter, you will only get the first notification when multiple updates are posted after the last time you read the topic. I suspect the user's notification setting is set for the "first time only".
• Make sure there is plenty of forum activity when testing... SMF relies on user activity to trigger background processing like notifications. Usually not an issue in production, but a big issue during testing where you may be the only active user. You need to simulate activity.

[jindroush]

Can you please elaborate on this? Because your first suspicion shouldn't have the effect at all.

I mean, from my point of view: 1) I set notification on board X 2) I set notification on topic Y in board X (this is the only topic in board X I have notifications set on) 3) my setting is 'immediate but only first'

Scenario A: 1) somebody posts to topic Z in board X -> I get notification (because of point 1) 2) THEN somebody posts to topic Y in board X -> I get the notification (because of point 2). I believe shortcutting here is wrong, as I believe the user expects the two notifications to be completely independent.

Scenario B: 1) somebody posts to topic Y in board X -> I get the notification (because of point 1) 2) again, somebody posts to Y -> no notification 3) somebody posts to Z in board X -> no notification

In scenario A, which is my user's scenario, it should not matter if the setting is Immediate/Immediate but only first.

[sbulen]

I think I confirmed the issue....

Some clarifications:

• A board level watch sends notifications only when a new topic is created. It does not send a notification any time anybody posts in any topic in the board.
• Email notifications are subject to some anti-spam measures. It will wait a few minutes, then only send one email to the user. It eats all dupes. E.g., if 20 replies are posted to topic Y, and 3 new topics are created in board X, within one minute, you may only get one email notification.
• As a result, you should never expect a notification for every single update. But you should expect at least 1 each time the conditions are met.

So, in your scenario A, that is true if the post to Z was a new topic. Two notifications are sent. If Z was a pre-existing topic, and someone replied to it, there would be no notification sent.

When testing these scenarios, you should wait sufficient time has passed so the dupe email detection isn't a factor. Wait until you receive the email before doing the second post.

Scenario B, however, fails.... It looks like when a topic notification is sent, it is setting both the topic & board 'sent' flags. It should only set the topic 'sent' flag.

So, yes, confirmed.

[sbulen]

Note parallel discussion in the boards: https://www.simplemachines.org/community/index.php?topic=584846.0

[nc3man]

Thanks for the explanation regarding the delay "feature." That makes sense for a forum with a large number of subscribers and active discussions. Our use case is completely different, however. I would like an option so that immediate is truly "immediate." Default behavior could remain as is, but with an administrative option to update the notification queue asap.

Why? We only have about 200 members in our bike club of which only ~40 are active in the forum. Our posting frequency is rather sparse, but very important alerting everyone of new events and interesting material relating to bicycling. We may typically have about ten posts, including replies, per week. Our issue is when a bike ride may need to be cancelled at the last minute, usually due to unexpected weather by a ride leader who posts to the forum, expecting anyone watching the forum to get the message right away. We also use a 3rd party mail service for all bulk emails and have that configured for SMF to use to avoid spamming, in addition to only allowing validated club members to even post (world can read).

On Sun, Feb 26, 2023, 6:20 PM sbulen @.***> wrote:

I think I confirmed the issue....

Some clarifications:

• A board level watch sends notifications only when a new topic is created. It does not send a notification any time anybody posts in any topic in the board.
• Email notifications are subject to some anti-spam measures. It will wait a few minutes, then only send one email to the user. It eats all dupes. E.g., if 20 replies are posted to topic Y, and 3 new topics are created in board X, within one minute, you may only get one email notification.
• As a result, you should never expect a notification for every single update. But you should expect at least 1 each time the conditions are met.

So, in your scenario A, that is true if the post to Z was a new topic. If Z was a pre-existing topic, and someone replied to it, there would be no notification sent.

When testing these scenarios, you should wait sufficient time has passed so the dupe email detection isn't a factor. Wait until you receive the email before doing the second post.

Scenario B, however, fails.... It looks like when a topic notification is sent, it is setting both the topic & board 'sent' flags. It should only set the topic 'sent' flag.

So, yes, confirmed.

— Reply to this email directly, view it on GitHub https://github.com/SimpleMachines/SMF/issues/7680#issuecomment-1445574263, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGW62RPJDTGU455UE2SIZTWZQFN3ANCNFSM6AAAAAAUGUVZCI . You are receiving this because you commented.Message ID: @.***>

[sbulen]

Note that the email notifications will always go out upon an update when the frequency is set to "Immediate". Typically within minutes. That is sufficient for almost all use cases.

Since the notification process is usage driven, for very low usage forums, some folks call cron.php as a true cron task every few minutes. This would guarantee that the notification would never exceed a few minutes. If interested in that, check the forum over at simplemachines.org.

[nc3man]

Thanks @sbulen for the cron suggestion. I searched the community and saw some issues with using cron.php directly, but there was a clever reply to set up cron to simply open the forum home page (to /dev/null) periodically in the regular server cron. I'll try running that every 5 minutes unless you have a better recommendation. It seems like adding another item under Maintenance > Scheduled Tasks for minimum time period between flushing the notification queue would be handy (for example CiviCRM has such a setting for flushing its mail queue). Waiting for someone to access the forum before notifications are sent is an odd design choice.

[Sesquipedalian]

I searched the community and saw some issues with using cron.php directly, but there was a clever reply to set up cron to simply open the forum home page (to /dev/null) periodically in the regular server cron. I'll try running that every 5 minutes unless you have a better recommendation.

There really shouldn't be any need for such a roundabout process, and it would be more likely to break than just executing cron.php directly. Just set your cron job to run /path/to/php -f /path/to/cron.php every few minutes. (Note: you shouldn't run it more often than every five minutes. It won't break any of SMF's bundled background tasks if you do, but it will be wasteful, and it's possible that background tasks added by mods could actually break.)

Waiting for someone to access the forum before notifications are sent is an odd design choice.

It's designed that way because not all forum administrators will have the necessary permissions (and/or knowledge) to add jobs to the server's crontab. The "webcron" system allows background tasks to be completed even in such cases.

The ideal setup is to use real cron, or even both real cron and "webcron" together if you want maximum speed and have the server resources.

[sbulen]

I actually don't think wget triggers .js... It will only copy the html IIRC. I'd confirm cron.php is actually getting triggered...

I just have cron invoking cron.php directly in one of my envs. Works fine.

Note not all OSs support cron. SMF needs to work ootb whether cron is available & authorized or not.

[nc3man]

Got it. Thanks @sbulen / @Sesquipedalian . I changed my 5 minute crontab to directly use php to execute /path/smf/cron.php. It runs without error. That referenced suggestion to use wget should be corrected.

[nc3man]

Although this issue is closed (Thanks!), in another post @Arantor confirmed that wget will trigger a single page load and nudge the mail queue to flush. It could take several page loads to clear a couple hundred notifications in the mail queue. I did confirm that also adding cron.php to our crontab does indeed take care of adding to the queue via CreatePost-Notify.php, but it appears to not really take action to drain the queue. This is important for all small forums with relatively low activity, but nevertheless a strong requirement for prompt notification. That's why the people who don't visit a lot, set a watch in the first place.