search

freescout-help-desk / freescout

FreeScout — Free self-hosted help desk & shared mailbox (Zendesk / Help Scout alternative)
https://freescout.net
GNU Affero General Public License v3.0
3.05k stars 502 forks source link

Cannot send mail via SMTP #4253

Closed sr-fias closed 1 month ago

sr-fias commented 1 month ago

PHP version: 8.2.22 FreeScout version: 1.8.153 Database: MySQL Are you using CloudFlare: No Are you using non-official modules: No

We are currently testing FreeScout via the Docker Container provided by tiredofit.

The system is up and running and IMAP connection to the mailbox is working fine and fetching mail.

But we have an issue with sending mail. Testing the settings always leads to a generic error in a red box:

Failed to authenticate on SMTP server with username "*********" using 0 possible authenticators.

Server, username and password are the same between SMTP and IMAP. SMTP port is 587.

One thing that may be uncommon in our setup: The authentication username differs from the email address.

Additionally below the test field a yellow box shows the following SMTP connection log:

++ Starting Swift_SmtpTransport
<< 220 ****************** Dovecot (Ubuntu) ready.

>> EHLO [127.0.0.1]

<< 250-******************
250-8BITMIME
250-AUTH 
250-BURL imap
250-CHUNKING
250-ENHANCEDSTATUSCODES
250-SIZE
250-STARTTLS
250 PIPELINING

>> STARTTLS

<< 220 2.0.0 Begin TLS negotiation now.

>> EHLO [127.0.0.1]

<< 250-******************
250-8BITMIME
250-AUTH PLAIN
250-BURL imap
250-CHUNKING
250-ENHANCEDSTATUSCODES
250-SIZE
250 PIPELINING

If we leave the username field empty we get a longer error in a red box:

Expected response code 250 but got code "530", with message "530 5.7.0 Authentication required."
Log data:

followed by a long string of connection logs with multiple retries of sending a mail and authentication errors inbetween. This was done to verify that the connection to the SMTP server is actually working.

So the connection to the SMTP server is functioning but somehow the authentication using username/password is failing. Same username/password works for IMAP and in Thunderbird, which is why we expect some sort of bug with FreeScout.

Can anyone help us out?

Thanks!

freescout-help commented 1 month ago

You'll need to figure it out by investigating the interaction between FreeScout and you mail server.

sr-fias commented 1 month ago

Mail server logs the following:

2024-09-27T09:11:49.872945+02:00 **** dovecot: submission-login: Disconnected: Connection closed (no auth attempts in 0 secs): user=<>, rip=162.x.x.x, lip=10.x.x.x, session=<uKehkRQjuICiN6WA>
2024-09-27T09:11:49.976086+02:00 **** dovecot: submission-login: Disconnected: Connection closed (no auth attempts in 0 secs): user=<>, rip=162.x.x.x, lip=10.x.x.x, TLS, session=<WjujkRQjvoCiN6WA>

I cannot identify any other helpful logs within FreeScout. The mail server logs indicate to me that the issues lies within FreeScout: The user field is empty, whereas a successful submission to the same user account via thunderbird works fine:

2024-09-27T09:22:35.150669+02:00 **** dovecot: submission-login: proxy(************): Started proxying to ***** (0.076 secs): user=<*********>, method=PLAIN, rip=x.x.x.x, lip=10.x.x.x, TLS, session=<KZ8WuBQjyOIKUw12>
KlausdB commented 1 month ago

Not sure if this will be helpful or not as I've never used Dovecot for email submission - always postfix before it. Dovecot uses the BURL protocol to enable the submission process. BURL uploads the message to the Sent folder and then submits it. The process can be problematic with different clients - yet Thunderbird does support it. I haven't found a lot in relation to support in the SwiftMailer client that FreeScout uses.

The 'clue' in your errors though could be "0 authentication methods". Check the Dovecot auth methods in its conf file. You may want to ensure it includes CRAM-MD5 - eg: auth_mechanisms = plain login cram-md5 in /etc/dovecot/conf.d/10-auth.conf This refers for dovecot 2.3: https://doc.dovecot.org/2.3/configuration_manual/authentication/authentication_mechanisms/

hope that at least points you in the right direction.

sr-fias commented 1 month ago

@KlausdB 💯

You were right on the money! I hardcoded the authentication method plain in the setMailDriver function where the SMTP driver is configured.

Specifically I added \Config::set('mail.auth_mode', 'PLAIN'); after the username and password are set in line 148.

Immediately the testmail ist sent and received.

Our dovecot was not configured to accept the default auth mode login as is set in config/mail.php.

Changing this config to plain works well!

@freescout-help Maybe make the login mechanism configurable as well?

SergeyMi88 commented 1 month ago

Hello, we have encountered a similar problem. Our support is currently working on version 1.8.140. We have been trying to update to new release versions for several months, but after the update, mail stops working with the error:

++ Starting Swift_SmtpTransport
<< 220 ****************** ESMTP Postfix

>> EHLO [127.0.0.1]

<< 250-*****************
250-PIPELINING
250-SIZE 20000000
250-STARTTLS
250-AUTH PLAIN
250-AUTH=PLAIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN

App log:

Failed to authenticate on SMTP server with username "**" using 0 possible authenticators. {"exception":"[object] (Swift_TransportException(code: 0): Failed to authenticate on SMTP server with username \"\" using 0 possible authenticators. at /www/html/overrides/swiftmailer/swiftmailer/lib/classes/Swift/Transport/Esmtp/AuthHandler.php:191)

Our administrator tried to make changes to the files described above, but this did not help to solve the problem. Perhaps you can suggest some other options for resolving this situation?

PHP version: 8.2.22 FreeScout version: 1.8.153 Database: Mysql (10.6.7-MariaDB) Are you using CloudFlare: No Are you using non-official modules: No

KlausdB commented 1 month ago

@SergeyMi88 You would likely need to adjust the AUTH methods on your mail server instead of freescout. Can you post details/specs of that server?

SergeyMi88 commented 1 month ago

@KlausdB We use Postfix mail server. Server settings:

smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous
smtpd_sasl_type = dovecot
smtpd_sasl_path = dovecot-auth
smtpd_tls_CApath = /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem

Restrictions:

smtpd_helo_restrictions = permit_mynetworks,
                          permit_sasl_authenticated,
                          reject_non_fqdn_hostname,
                          reject_invalid_hostname,
                          permit

smtpd_client_restrictions = permit_sasl_authenticated,
                          permit_mynetworks,
                          # check_client_access hash:/etc/postfix/amavis_bypass,
                          # check_client_access hash:/etc/postfix/client_access,
                          # reject_rbl_client bl.spamcop.net,
                          reject_unknown_client

I would like to draw your attention to the fact that FreeScout and our mail server worked perfectly for a long time. The problems started after the release of version 1.8.141. We tried to install several more subsequent releases that mentioned changes in the mail operation, but the problem remained unresolved. As a result, we had to roll back to version 1.8.140, which is what our main project is currently running on.

freescout-help commented 1 month ago

@SergeyMi88 There was no changes made in the emails sending subsystem - it was using Swiftmail library to send emails before and it's using it now.

KlausdB commented 1 month ago

OK @SergeyMi88 I can see that you are telling Postfix to use Dovecot for authentication, so the issue appears the same as the previous user who resolved theirs in the freescout code instead of the server. Their instance was talking to Dovecot direct however. Two things. First up look at the Dovecot config on your mail server as per the previous advice:
Check the Dovecot auth methods in its conf file. You may want to ensure it includes CRAM-MD5 - eg: auth_mechanisms = plain login cram-md5 in /etc/dovecot/conf.d/10-auth.conf This refers for dovecot 2.3: https://doc.dovecot.org/2.3/configuration_manual/authentication/authentication_mechanisms/

If not that, check the dovecot configuration for auth. Is it listening and responding correctly to postfix. Is there a: service auth { ... etc statement? Does that point to a valid path?

If that is problematic, you could look at setting up postfix to auth smtp such as here: https://www.cyberciti.biz/faq/postfix-smtp-authentication-for-mail-servers/

Note that I have nothing to do with Freescout, just a user trying to help. I'm on version 1.8.153 and looking back through the changelog suggests there were some encryption method changes made at your release level. It's not specifically for smtp, but can effect it.

sr-fias commented 1 month ago

@freescout-help I have identified three changes between 1.8.140 and 1.8.153 that could cause changes in behaviour.

  1. Hardcoded auth_mode to "login" in overrides/swiftmailer/swiftmailer/lib/classes/Swift/SmtpTransport.php
  2. Added check for auth_mode config in overrides/laravel/framework/src/Illuminate/Mail/TransportManager.php
  3. Added config option auth_mode in config/mail.php

My guess is that one of these changes also changed the default authentication methods that Swiftmailer will use to try and establish a connection -- in particular setting the authentication methods to login only.

freescout-help commented 1 month ago

@sr-fias You are right.

@SergeyMi88 Try to change this line https://github.com/freescout-help-desk/freescout/blob/dist/overrides/swiftmailer/swiftmailer/lib/classes/Swift/SmtpTransport.php#L42 to the following:

        if ($auth_mode == 'XOAUTH2') {
            $this->setAuthMode($auth_mode);
        }
SergeyMi88 commented 1 month ago

We made this change on our test and main server and it helped, mail works without errors. Thanks to everyone for the help. @freescout-help Can we expect this fix in the next releases?

sr-fias commented 1 month ago

@SergeyMi88 should be fixed in 0280e4f