Mail queue purged when SMTP authentication failed

[ThibautPlg]

Hello, I've noticed an unexpected behavior regarding the mail notification. When the SMTP authentication is resulting in an error, the mail is purged instead of being kept and sent later.

Steps to reproduce

1. Set on purpose a wrong SMTP password
2. Generate an activity (adding a user to a group for example)
3. Notice that a new entry is made in oc_activity_mq

   +---------+---------------+-----------------+----------------+------------------+-----------+-------------+--------------------------------------+-------------+-----------+
   | mail_id | amq_timestamp | amq_latest_send | amq_type       | amq_affecteduser | amq_appid | amq_subject | amq_subjectparams                    | object_type | object_id |
   +---------+---------------+-----------------+----------------+------------------+-----------+-------------+--------------------------------------+-------------+-----------+
   |   28760 |    1713793593 |      1713797193 | group_settings | johndoe          | settings  | group_added | {"user":"johndoe","group":"mygroup"} |             |         0 |
   +---------+---------------+-----------------+----------------+------------------+-----------+-------------+--------------------------------------+-------------+-----------+

4. Trigger the sendEmail function using occ activity:send-mails
5. The oc_activity_mq table is now empty
6. I have a log error not caught from the Activity App but from symfony directly

Expected behaviour

The mail queue shouldn't be purged and my mail should be postponed until the SMTP password is corrected.

Actual behaviour

The mail queue is purged and my mail lost.

From my limited understanding, the smtp error should be caught by this code: https://github.com/nextcloud/activity/blob/master/lib/MailQueueHandler.php#L364, however the string Failed sending activity email to user is nowhere to be seen in my logs. Neither are the other error strings I could find on the codebase (missing email, invalid email address, parser error...)

Server configuration

• Nextcloud version: 27.1.7.2
• Activity version: 2.19.0
• Loglevel 0 so I can see the debug logs

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...

• Yes, LDAP

  Logs

Nextcloud log (data/nextcloud.log)

{
    "reqId": "F6qFnlDccP6yXjSMFtWZ",
    "level": 0,
    "time": "2024-04-22T16:03:16+02:00",
    "remoteAddr": "",
    "user": "--",
    "app": "core",
    "method": "",
    "url": "--",
    "message": "Sending mail to \"Array\n(\n    [mymail@mailproviderexample.com] => John Doe\n)\n\" with subject \"Activity at [testing] nextcloud\" failed",
    "userAgent": "--",
    "version": "27.1.7.2",
    "exception": {
        "Exception": "Symfony\\Component\\Mailer\\Exception\\TransportException",
        "Message": "Failed to authenticate on SMTP server with username \"nextcloud-test-testing@nextcloud.mailproviderexample.com\" using the following authenticators: \"LOGIN\". Authenticator \"LOGIN\" returned \"Expected response code \"235\" but got code \"535\", with message \"535 5.7.3 Authentication unsuccessful\".\".",
        "Code": 0,
        "Trace": [
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/mailer/Transport/Smtp/EsmtpTransport.php",
                "line": 123,
                "function": "handleAuth",
                "class": "Symfony\\Component\\Mailer\\Transport\\Smtp\\EsmtpTransport",
                "type": "->",
                "args": [
                    [
                        "gssapi",
                        "ntlm",
                        "login"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php",
                "line": 253,
                "function": "doHeloCommand",
                "class": "Symfony\\Component\\Mailer\\Transport\\Smtp\\EsmtpTransport",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php",
                "line": 194,
                "function": "start",
                "class": "Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/mailer/Transport/AbstractTransport.php",
                "line": 72,
                "function": "doSend",
                "class": "Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Mailer\\SentMessage"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/mailer/Transport/Smtp/SmtpTransport.php",
                "line": 136,
                "function": "send",
                "class": "Symfony\\Component\\Mailer\\Transport\\AbstractTransport",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Mailer\\SentMessage"
                    ],
                    [
                        "Symfony\\Component\\Mailer\\DelayedEnvelope"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/mailer/Mailer.php",
                "line": 45,
                "function": "send",
                "class": "Symfony\\Component\\Mailer\\Transport\\Smtp\\SmtpTransport",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Mime\\Email"
                    ],
                    null
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/lib/private/Mail/Mailer.php",
                "line": 216,
                "function": "send",
                "class": "Symfony\\Component\\Mailer\\Mailer",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Mime\\Email"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/apps/activity/lib/MailQueueHandler.php",
                "line": 415,
                "function": "send",
                "class": "OC\\Mail\\Mailer",
                "type": "->",
                "args": [
                    [
                        "OC\\Mail\\Message"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/apps/activity/lib/MailQueueHandler.php",
                "line": 165,
                "function": "sendEmailToUser",
                "class": "OCA\\Activity\\MailQueueHandler",
                "type": "->",
                "args": [
                    "johndoe",
                    "mymail@mailproviderexample.com",
                    "en",
                    "Europe/Paris",
                    1713794585
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/apps/activity/lib/Command/SendEmails.php",
                "line": 110,
                "function": "sendEmails",
                "class": "OCA\\Activity\\MailQueueHandler",
                "type": "->",
                "args": [
                    500,
                    1713794585,
                    true,
                    null
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/console/Command/Command.php",
                "line": 298,
                "function": "execute",
                "class": "OCA\\Activity\\Command\\SendEmails",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Console\\Input\\ArgvInput"
                    ],
                    [
                        "Symfony\\Component\\Console\\Output\\ConsoleOutput"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/core/Command/Base.php",
                "line": 177,
                "function": "run",
                "class": "Symfony\\Component\\Console\\Command\\Command",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Console\\Input\\ArgvInput"
                    ],
                    [
                        "Symfony\\Component\\Console\\Output\\ConsoleOutput"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/console/Application.php",
                "line": 1040,
                "function": "run",
                "class": "OC\\Core\\Command\\Base",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Console\\Input\\ArgvInput"
                    ],
                    [
                        "Symfony\\Component\\Console\\Output\\ConsoleOutput"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/console/Application.php",
                "line": 301,
                "function": "doRunCommand",
                "class": "Symfony\\Component\\Console\\Application",
                "type": "->",
                "args": [
                    [
                        "OCA\\Activity\\Command\\SendEmails"
                    ],
                    [
                        "Symfony\\Component\\Console\\Input\\ArgvInput"
                    ],
                    [
                        "Symfony\\Component\\Console\\Output\\ConsoleOutput"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/console/Application.php",
                "line": 171,
                "function": "doRun",
                "class": "Symfony\\Component\\Console\\Application",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Console\\Input\\ArgvInput"
                    ],
                    [
                        "Symfony\\Component\\Console\\Output\\ConsoleOutput"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/lib/private/Console/Application.php",
                "line": 218,
                "function": "run",
                "class": "Symfony\\Component\\Console\\Application",
                "type": "->",
                "args": [
                    [
                        "Symfony\\Component\\Console\\Input\\ArgvInput"
                    ],
                    [
                        "Symfony\\Component\\Console\\Output\\ConsoleOutput"
                    ]
                ]
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/console.php",
                "line": 100,
                "function": "run",
                "class": "OC\\Console\\Application",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/nextcloud/my-nextcloud-test.com/occ",
                "line": 11,
                "args": [
                    "/var/www/html/nextcloud/my-nextcloud-test.com/console.php"
                ],
                "function": "require_once"
            }
        ],
        "File": "/var/www/html/nextcloud/my-nextcloud-test.com/3rdparty/symfony/mailer/Transport/Smtp/EsmtpTransport.php",
        "Line": 198,
        "message": "Sending mail to \"Array\n(\n    [mymail@mailproviderexample.com] => John Doe\n)\n\" with subject \"Activity at [testing] nextcloud\" failed",
        "exception": {},
        "CustomMessage": "Sending mail to \"Array\n(\n    [mymail@mailproviderexample.com] => John Doe\n)\n\" with subject \"Activity at [testing] nextcloud\" failed"
    }
}

Best regards,

[ThibautPlg]

Hi Any news on this? Can someone reproduce or give a insight?

[joshtrichards]

The cause appears to be that even though we appear to anticipate the failure here:

https://github.com/nextcloud/activity/blob/50037ac539f02b3c10fd9778c2ba0c05d839c9e0/lib/MailQueueHandler.php#L414-L423

The exception from the Mailer (which we're calling there) is caught before it gets back to the above code path. That happens here:

https://github.com/nextcloud/server/blob/7a6508b853a484340d9c62068ef99723617467eb/lib/private/Mail/Mailer.php#L215-L235