[Bug]: SMTP failure after upgrade from 1.37 to 1.51

[cah-dsaar]

What happened?

We updated our FusionAuth-hosted instance last night from version 1.37 to version 1.51 because of the XSS vulnerability in the FusionAuth admin application.

Prior to the upgrade, our STMP integration was operable, afterwards it is not. We have verified that the email provider is accessible (our app's backend uses the same provider), and the configuration does not seem to have changed, using port 587 and TLS. The email provider is paubox, so the host is smtp.paubox.com.

Sending a test email from the tenant editing screen produces: Unable to send email via JavaMail

Prime Messaging Exception Exception reading response Cause: SocketTimeoutException: Read timed out

Triggering a welcome email to be sent results in the following in the Event Log (Debug is enabled) Async Email Send exception occurred.

Template Id: 006c9493-53c7-4e74-9332-**** Template Name: * - Welcome Email - Dev Tenant Id: 00000000-0000-0000-7661-** Addressed to: *

Cause: jakarta.mail.MessagingException : Message: Exception reading response

App Log is reporting: DEBUG: Jakarta Mail version 2.1.2 DEBUG: URL jar:file:/usr/local/fusionauth/fusionauth-app/lib/smtp-2.0.2.jar!/META-INF/javamail.providers DEBUG: successfully loaded resource: jar:file:/usr/local/fusionauth/fusionauth-app/lib/smtp-2.0.2.jar!/META-INF/javamail.providers DEBUG: Tables of loaded providers DEBUG: Providers Listed By Class Name: {org.eclipse.angus.mail.smtp.SMTPTransport=jakarta.mail.Provider[TRANSPORT,smtp,org.eclipse.angus.mail.smtp.SMTPTransport,Oracle], org.eclipse.angus.mail.smtp.SMTPSSLTransport=jakarta.mail.Provider[TRANSPORT,smtps,org.eclipse.angus.mail.smtp.SMTPSSLTransport,Oracle]} DEBUG: Providers Listed By Protocol: {smtp=jakarta.mail.Provider[TRANSPORT,smtp,org.eclipse.angus.mail.smtp.SMTPTransport,Oracle], smtps=jakarta.mail.Provider[TRANSPORT,smtps,org.eclipse.angus.mail.smtp.SMTPSSLTransport,Oracle]} DEBUG: successfully loaded resource: /META-INF/javamail.default.address.map DEBUG: URL jar:file:/usr/local/fusionauth/fusionauth-app/lib/smtp-2.0.2.jar!/META-INF/javamail.address.map DEBUG: successfully loaded resource: jar:file:/usr/local/fusionauth/fusionauth-app/lib/smtp-2.0.2.jar!/META-INF/javamail.address.map DEBUG: setDebug: Jakarta Mail version 2.1.2

Thinking there was an upgrade to JavaMail that changed some defaults related to TLS, I added various combinations of the following to SMTP settings, to no avail:

mail.debug=true mail.smtp.auth=true mail.smtp.starttls.enable=true mail.smtp.starttls.required=true mail.smtp.ssl.protocols=TLSv1.2

(Paubox documentation says STARTTLS and TLSv1.2.)

SWAKS succeeds: === Trying smtp.paubox.com:587... === Connected to smtp.paubox.com. <- 220 welcome to paubox smtp -> EHLO ip----.ec2.internal <- 250-paubox smtp at your service <- 250-8BITMIME <- 250-SMTPUTF8 <- 250-PIPELINING <- 250-AUTH LOGIN PLAIN <- 250-STARTTLS <- 250 OK -> STARTTLS <- 220 Ready to start TLS === TLS started with cipher TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256 === TLS no local certificate set === TLS peer DN="/CN=paubox.com" ~> EHLO ip----.ec2.internal <~ 250-paubox smtp at your service <~ 250-8BITMIME <~ 250-SMTPUTF8 <~ 250-PIPELINING <~ 250-AUTH LOGIN PLAIN <~ 250 OK ~> AUTH LOGIN <~ 334 **** ~> **** <~ 334 **** ~> **** <~ 235 OK ~> MAIL FROM:***************@***************.com <~ 250 OK ~> RCPT TO:***.****@**********.com <~ 250 OK ~> DATA <~ 354 Enter message, ending with "." on a line by itself ~> Date: Wed, 24 Jul 2024 22:29:19 +0000 ~> To: *.@**.com ~> From: *****@***.com ~> Subject: test Wed, 24 Jul 2024 22:29:19 +0000 ~> Message-Id: **************.******@ip-**-*-*-**.ec2.internal ~> X-Mailer: swaks v20201014.0 jetmore.org/john/code/swaks/ ~> ~> Please disregard! ~> ~> ~> . <~ 250 Requested mail action okay, completed ~> QUIT <~ 221 Service closing transmission channel === Connection closed with remote host.

Version

1.51.2

Affects Versions

No response

[cah-dsaar]

Added values for these two timeouts in SMTP Advanced Settings. Perhaps default values were changed at some point, but that is not documented in release notes.

mail.smtp.timeout mail.smtp.connectiontimeout

[mooreds]

Looks like the change happened in 1.44.0 and was in the release notes, but maybe the recommended action was incomplete?

https://fusionauth.io/docs/release-notes/#version-1-44-0

Add default configuration for read and connect timeouts to the SMTP server configuration. This helps protect FusionAuth against an SMTP server that never closes a socket. From time to time we observed an SMTP server hold open a socket, and tie up a send thread which may block other senders.

[cah-dsaar]

Yes, document the values of the default configuration.

Catching and logging the timeout exceptions would be helpful. In this case, the "Debug Enabled" SMTP setting didn't log anything, presumably because it didn't get connected enough. It would have been helpful to see "Connecting" and "Timeout" in the logs

[mooreds]

@cah-dsaar thanks so much for your feedback. We have an open PR to document these values and I opened a separate issue (https://github.com/FusionAuth/fusionauth-issues/issues/2844) for your request for additional debugging information.

I plan to close this issue out once the documentation is approved and merged.

[mooreds]

Closing because the doc PR is merged (should be live on the site in 5 or so min).