SMTP e-mails no longer working

obs0lete commented 4 years ago

Subject of the issue

Your environment

Bitwarden_rs version: Version: 1.14.2-4eee6e7a
Install method: Docker
Clients used: N/A
Reverse proxy and version: N/A
Version of mysql/postgresql: N/A
Other relevant information: This is a docker container being run on a Raspberry Pi 4.

Steps to reproduce

Create a new docker container as shown below:

docker run \
-p 7004:80 \
-e ADMIN_TOKEN=something\
-v bitwarden:/config \
--name bitwarden_rs \
--restart always \
bitwardenrs/server:raspberry

After image is deployed, I go to the Admin page and enter my SMTP details, using Office 365 credentials, then save the changes.

Expected behaviour

I should get an e-mail to let me know it is working.

Actual behaviour

I get a pop-up that says:

Error sending SMTP test email
SmtpError

The e-mails are being sent as user1@email.com, but are actually sent from user2@email.com. user1@email.com does have the Send As permission enabled. This was also working just fine in the previous release of Bitwarden_rs.

SMTP Setup as follows:
Host: smtp.office365.com
Enable SSL: true
Use explicit TLS: false
Port: 587
From Address: user1@email.com
From Name: user1@email.com
Username: user2@email.com
Password: something
Json form auth mechanism: empty
SMTP connection timeout: 15

Relevant logs

[2020-05-29 14:14:11][error][ERROR] SmtpError.
[CAUSE] Permanent(
    Response {
        code: Code {
            severity: PermanentNegativeCompletion,
            category: MailSystem,
            detail: Four,
        },
        message: [
            "5.2.0 STOREDRV.Submission.Exception:SendAsDeniedException.MapiExceptionSendAsDenied; Failed to process message due to a permanent exception with message Cannot submit message. 0.35250:0A007281, 1.36674:01000000, 1.61250:00000000, 1.45378:02000000, 1.44866:00000000, 1.36674:7A000000, 1.61250:00000000, 1.45378:05000000, 1.44866:00140000, 1.36674:0A000000, 1.61250:00000000, 1.45378:16000000, 1.44866:CF4D0000, 1.36674:0E000000, 1.61250:00000000, 1.45378:E84D0000, 1.44866:EC010000, 16.55847:391C0000, 17.43559:0000000030020000000000000100000000000000, 20.52176:140FE58B0E00F01F1F001432, 20.50032:140FE58B7E17F01F07000000, 0.35180:1F000130, 255.23226:D32F0000, 255.27962:7A000000, 255.27962:0A000000, 255.27962:0E000000, 255.31418:35000000, 0.35250:0A001336, 1.36674:0A000000, 1.61250:00000000, 1.45378:02000000, 1.44866:3C000000, 1.36674:32000000, 1.61250:00000000, 1.45378:41000000, 1.44866:01000000, 16.55847:A8000000, 17.43559:0000000070030000000000000700000000000000, 20.52176:140FE58B0E0010106E000000, 20.50032:140FE58B7E1700007A300000, 0.35180:73000000, 255.23226:0A000C80, 255.27962:0A000000, 255.27962:32000000, 255.17082:DC040000, 0.27745:B0300000, 4.21921:DC040000, 255.27962:FA000000, 255.1494:E1300000, 0.38698:05000780, 0.37692:01000000, 0.37948:0F010480, 5.33852:00000000534D545000000100, 7.36354:010000000000010924000100, 4.56248:DC040000, 7.40748:010000000000010B00000000, 7.57132:000000000000000061676520, 1.63016:32000000, 4.39640:DC040000, 8.45434:DFC6D02BAEDC3249BC83EB5ECE6ECB3009010480, 5.10786:0000000031352E32302E333034352E3031343A595442505230314D42323733363A65313039653630642D646236622D343939622D383664662D3865346666636461396362330050A000000000, 7.51330:9FF03E8FDA03D80831343A59, 0.39570:30314D42, 1.55954:0A000000, 0.49266:02000000, 1.33010:0A000000, 2.54258:00000000, 0.40002:7A000000, 1.56562:00000000, 1.64146:32000000, 1.33010:32000000, 2.54258:DC040000, 255.1750:BE000000, 255.31418:0A006136, 0.22753:B3320000, 255.21817:DC040000, 0.64418:03003866, 4.39842:DC040000, 0.41586:C8000000, 4.60547:DC040000, 0.21966:BD320000, 4.30158:DC040000 [Hostname=YTBPR01MB2736.CANPRD01.PROD.OUTLOOK.COM]",
        ],
    },
)
[2020-05-29 14:14:11][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

BlackDex commented 4 years ago

Seems like office is blocking this message. Could you try to use TLS instead of SSL?

obs0lete commented 4 years ago

Seems like office is blocking this message. Could you try to use TLS instead of SSL?

Changing to TLS gives the following results:

[2020-05-29 15:15:22][error][ERROR] SmtpError.
[CAUSE] Client(
    "No compatible authentication mechanism was found",
)
[2020-05-29 15:15:22][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Using this page as a reference (https://support.office.com/en-us/article/pop-and-imap-email-settings-for-outlook-8361e398-8af4-4e97-b147-6c6c4ac95353) I also tried using the following scenarios:

Host: outlook.office365.com
Port: 993
Enable SSL: true
Use explicit TLS: false

Result:
[2020-05-29 15:19:42][error][ERROR] SmtpError.
[CAUSE] Io(
    Os {
        code: 11,
        kind: WouldBlock,
        message: "Resource temporarily unavailable",
    },
)
[2020-05-29 15:19:42][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Host: outlook.office365.com
Port: 993
Enable SSL: false
Use explicit TLS: true

Result:
[2020-05-29 15:21:10][error][ERROR] SmtpError.
[CAUSE] Io(
    Os {
        code: 11,
        kind: WouldBlock,
        message: "Resource temporarily unavailable",
    },
)
[2020-05-29 15:21:10][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Host: outlook.office365.com
Port: 993
Enable SSL: true
Use explicit TLS: true

Result:
[2020-05-29 15:24:50][error][ERROR] SmtpError.
[CAUSE] Io(
    Os {
        code: 11,
        kind: WouldBlock,
        message: "Resource temporarily unavailable",
    },
)
[2020-05-29 15:24:50][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Host: outlook.office365.com
Port: 995
Enable SSL: false
Use explicit TLS: true

Result:
[2020-05-29 15:22:30][error][ERROR] SmtpError.
[CAUSE] Io(
    Os {
        code: 11,
        kind: WouldBlock,
        message: "Resource temporarily unavailable",
    },
)
[2020-05-29 15:22:30][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Host: outlook.office365.com
Port: 995
Enable SSL: true
Use explicit TLS: false

Result:
[2020-05-29 15:23:36][error][ERROR] SmtpError.
[CAUSE] Io(
    Os {
        code: 11,
        kind: WouldBlock,
        message: "Resource temporarily unavailable",
    },
)
[2020-05-29 15:23:36][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Host: outlook.office365.com
Port: 995
Enable SSL: true
Use explicit TLS: true

Result:
[2020-05-29 15:26:03][error][ERROR] SmtpError.
[CAUSE] Io(
    Os {
        code: 11,
        kind: WouldBlock,
        message: "Resource temporarily unavailable",
    },
)
[2020-05-29 15:26:03][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Host: smtp.office365.com
Port: 587
Enable SSL: true
Use explicit TLS: false

Result:
[2020-05-29 15:27:49][error][ERROR] SmtpError.
[CAUSE] Permanent(
    Response {
        code: Code {
            severity: PermanentNegativeCompletion,
            category: MailSystem,
            detail: Four,
        },
        message: [
            "5.2.0 STOREDRV.Submission.Exception:SendAsDeniedException.MapiExceptionSendAsDenied; Failed to process message due to a permanent exception with message Cannot submit message. 0.35250:0A007281, 1.36674:0A000000, 1.61250:00000000, 1.45378:02000000, 1.44866:A64D0000, 1.36674:0E000000, 1.61250:00000000, 1.45378:AB4D0000, 1.44866:EC010000, 16.55847:DA1B0000, 17.43559:0000000024020000000000000000000000000000, 20.52176:140FE58B0E00F01F00000000, 20.50032:140FE58B7E17F01F00000000, 0.35180:140FE58B, 255.23226:0A007881, 255.27962:0A000000, 255.27962:0E000000, 255.31418:0A007981, 0.35250:0A00B230, 1.36674:0A000000, 1.61250:00000000, 1.45378:02000000, 1.44866:3C000000, 1.36674:32000000, 1.61250:00000000, 1.45378:41000000, 1.44866:01000000, 16.55847:A8000000, 17.43559:0000000070030000000000000100000000000000, 20.52176:140FE58B0E00101053000000, 20.50032:140FE58B7E1700006B300000, 0.35180:58000000, 255.23226:4800D13D, 255.27962:0A000000, 255.27962:32000000, 255.17082:DC040000, 0.27745:75300000, 4.21921:DC040000, 255.27962:FA000000, 255.1494:7A300000, 0.38698:05000780, 0.37692:01000000, 0.37948:30000000, 5.33852:00000000534D545000746F72, 7.36354:01000000000001094B2CAAA2, 4.56248:DC040000, 7.40748:010000000000010B00000000, 7.57132:00000000000000004D657373, 1.63016:32000000, 4.39640:DC040000, 8.45434:DFC6D02BAEDC3249BC83EB5ECE6ECB3002000000, 5.10786:0000000031352E32302E333034352E3031343A595442505230314D42323733363A65313039653630642D646236622D343939622D383664662D386534666663646139636233002E3031343A59, 7.51330:AB2588D8E403D8083A653130, 0.39570:642D6462, 1.55954:0A000000, 0.49266:56000000, 1.33010:0A000000, 2.54258:00000000, 0.40002:01000000, 1.56562:00000000, 1.64146:32000000, 1.33010:32000000, 2.54258:DC040000, 255.1750:AF000000, 255.31418:0A005D36, 0.22753:78320000, 255.21817:DC040000, 0.64418:0A00F565, 4.39842:DC040000, 0.41586:B9000000, 4.60547:DC040000, 0.21966:AE320000, 4.30158:DC040000 [Hostname=YTBPR01MB2736.CANPRD01.PROD.OUTLOOK.COM]",
        ],
    },
)
[2020-05-29 15:27:49][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

Host: smtp.office365.com
Port: 587
Enable SSL: false
Use explicit TLS: true

Result:
[2020-05-29 15:29:20][error][ERROR] SmtpError.
[CAUSE] Client(
    "No compatible authentication mechanism was found",
)
[2020-05-29 15:29:20][response][INFO] POST /admin/test/smtp (test_smtp) => 400 Bad Request

dprobinson commented 4 years ago

I'm wondering if this might be due to the VRFY command not being supported.

@obs0lete, to help debug this, can you try sending an email with this curl command, and paste us the output (don't forget to replace the PLACEHOLDERS and remove your password from the output!)

curl --url 'smtps://smtp.office365.com:465' --ssl-reqd --mail-from 'FROMUSER@YOURDOMAIN.TLD' --mail-rcpt 'RECIPIENT@YOURDOMAIN.TLD' --user 'SMTPUSERNAME:YOURPASSWORD' -v

obs0lete commented 4 years ago

Thanks for the reply!

The output was quite large, so I've shortened it a bit.

It starts:

* Expire in 0 ms for 6 (transfer 0xd04880)
* Expire in 1 ms for 1 (transfer 0xd04880)
* Expire in 0 ms for 1 (transfer 0xd04880)
[...]
* Expire in 50 ms for 1 (transfer 0xd04880)
* Expire in 50 ms for 1 (transfer 0xd04880)
*   Trying 40.101.128.18...
* TCP_NODELAY set
* Expire in 149898 ms for 3 (transfer 0xd04880)
* Expire in 200 ms for 4 (transfer 0xd04880)
*   Trying 2603:1036:c00:14::2...
* TCP_NODELAY set
* Expire in 149898 ms for 3 (transfer 0xd04880)
* Immediate connect fail for 2603:1036:c00:14::2: Network is unreachable
*   Trying 2603:1036:30a:800::2...
* TCP_NODELAY set
* Expire in 149898 ms for 3 (transfer 0xd04880)
* Immediate connect fail for 2603:1036:30a:800::2: Network is unreachable

...which eventually leads to:

* TCP_NODELAY set
* Expire in 149898 ms for 3 (transfer 0xd04880)
* Immediate connect fail for 2603:1036:30a:82f::2: Network is unreachable
* connect to 40.101.128.18 port 465 failed: Connection timed out

I noticed that this commaned was using port 465, but it should be 587 as per https://support.office.com/en-us/article/pop-and-imap-email-settings-for-outlook-8361e398-8af4-4e97-b147-6c6c4ac95353. So I adjusted the command as follows:

curl --url 'smtps://smtp.office365.com:587' --ssl-reqd --mail-from 'user1@email.com' --mail-rcpt 'user2@email.com' --user 'user2@email.com:user2PASSWORD' -v

...and I get this result:

[...]
* Expire in 2 ms for 1 (transfer 0x1b87880)
* Expire in 2 ms for 1 (transfer 0x1b87880)
*   Trying 40.101.128.18...
* TCP_NODELAY set
* Expire in 149996 ms for 3 (transfer 0x1b87880)
* Expire in 200 ms for 4 (transfer 0x1b87880)
* Connected to smtp.office365.com (40.101.128.18) port 587 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* error:1408F10B:SSL routines:ssl3_get_record:wrong version number
* Closing connection 0
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number

dprobinson commented 4 years ago

The server is using Explicit TLS (STARTTLS) on port 587, so I believe the correct settings should be:

Host: smtp.office365.com
Port: 587
Enable SSL: true
Use explicit TLS: false

N.B. The label for Explicit TLS is actually supposed to be Implicit TLS, as per bug #851

According to the link below, it appears there might be a bug at Microsoft's end preventing you sending from another email address: https://hi.service-now.com/kb_view.do?sysparm_article=KB0778441

obs0lete commented 4 years ago

The server is using Explicit TLS (STARTTLS) on port 587, so I believe the correct settings should be:
Host: smtp.office365.com
Port: 587
Enable SSL: true
Use explicit TLS: false
N.B. The label for Explicit TLS is actually supposed to be Implicit TLS, as per bug #851

According to the link below, it appears there might be a bug at Microsoft's end preventing you sending from another email address: https://hi.service-now.com/kb_view.do?sysparm_article=KB0778441

I tired your suggestion but I still get the same result. Even if I try sending email from the same account I still get the same results. Like I mentioned in my initial post, it was working just fine in the previous release.

On the same server, I have sSMTP running as my MTA and it is able to send e-mail from user2@email.com just fine using smtp.office365.com. Not exactly sure what's going on.

I just tried using a Gmail account and it seems to be working, meaning I'm getting the test e-mails. I'll have to leave it like this for now so at least I'm getting notifications.

BlackDex commented 4 years ago

@obs0lete I have tested it with my own office365.com account and it worked for me using the following settings (As configured via the /admin panel).

Host: smtp.office365.com
Enable SSL: true/checked
Use explicit TLS: false/unchecked
Port: 587
From Address: myname@mydomain.com
From Name: Bitwarden_RS
Username: myname@mydomain.com
Password: MyVerySecretPassword
Json form auth mechanism: "Login"

I Tested this twice, and worked without any issues.

obs0lete commented 4 years ago

@obs0lete I have tested it with my own office365.com account and it worked for me using the following settings (As configured via the /admin panel).
Host: smtp.office365.com
Enable SSL: true/checked
Use explicit TLS: false/unchecked
Port: 587
From Address: myname@mydomain.com
From Name: Bitwarden_RS
Username: myname@mydomain.com
Password: MyVerySecretPassword
Json form auth mechanism: "Login"
I Tested this twice, and worked without any issues.

Look at that, it's working! Thank you.

I think the problem I was having was that I was using an e-mail address in the From Name field. When I just changed this to another value it worked.

Thanks everyone for your help!

vdias commented 3 months ago

Outlook not working.

BlackDex commented 3 months ago

Outlook not working.

@vdias i would suggest to read this: https://github.com/dani-garcia/vaultwarden/discussions/4518#discussioncomment-9211666

Outlook stopped supporting plain logins.

dani-garcia / vaultwarden