search

imthenachoman / How-To-Secure-A-Linux-Server

An evolving how-to guide for securing a Linux server.
Creative Commons Attribution Share Alike 4.0 International
17.25k stars 1.1k forks source link

"Authentication Required" when trying to send a mail using Gmail #42

Closed gcoter closed 4 years ago

gcoter commented 4 years ago

Hello,

First of all, I want to thank you for this amazing tutorial. I learned a lot thanks to it :)

I followed your instructions to setup Exim4 on a Raspberry Pi server so that it can send mails using a Gmail account I created for this purpose. It has been working well for a while.

But for some reason, it stopped working last week. Looking at /var/log/exim4/mainlog, I can see logs like this each time I try to send a mail:

2019-10-19 15:30:30 1iLooQ-0002aM-3F H=smtp.gmail.com [2a00:1450:400c:c0b::6c] Network is unreachable
2019-10-19 15:30:30 1iLooQ-0002aM-3F ** <MY-GMAIL-ADDRESS> R=smarthost T=remote_smtp_smarthost H=smtp.gmail.com [64.233.184.108] X=TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256 CV=yes DN="C=US,ST=California,L=Mountain View,O=Google LLC,CN=smtp.gmail.com": SMTP error from remote mail server after pipelined MAIL FROM:<gcoter@localhost> SIZE=1410: 530-5.5.1 Authentication Required. Learn more at\n530 5.5.1  https://support.google.com/mail/?p=WantAuthError z13sm8095930wrq.51 - gsmtp
2019-10-19 15:30:32 1iLooS-0002aS-Q9 <= <> R=1iLooQ-0002aM-3F U=Debian-exim P=local S=2021
2019-10-19 15:30:32 1iLooQ-0002aM-3F Completed
2019-10-19 15:30:33 1iLooS-0002aS-Q9 ** <MY-GMAIL-ADDRESS> <gcoter@raspberrypi> R=smarthost T=remote_smtp_smarthost H=smtp.gmail.com [64.233.184.108] X=TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256 CV=yes DN="C=US,ST=California,L=Mountain View,O=Google LLC,CN=smtp.gmail.com": SMTP error from remote mail server after pipelined MAIL FROM:<> SIZE=3101: 530-5.5.1 Authentication Required. Learn more at\n530 5.5.1  https://support.google.com/mail/?p=WantAuthError u68sm10557199wmu.12 - gsmtp
2019-10-19 15:30:33 1iLooS-0002aS-Q9 Frozen (delivery error message)

So it seems like an authentication error. I followed the Google Support link which is written in the logs but it didn't help. Here is what I tried:

At this point, I don't understand what is wrong. I don't think it comes from the way I configured the server since it has been working for a long time. Is it possible that Google decided to prevent my server from sending mails?

hellresistor commented 4 years ago

Check the fail 2 ban log ? My fail2ban.log returns `2019-10-19 15:45:35,095 fail2ban.jail [6770]: INFO Initiated 'systemd' backend

2019-10-19 15:45:35,096 fail2ban.filter [6770]: ERROR No failure-id group in 'sendmail.*authentication failure (-13) SASL(-13):'

2019-10-19 15:45:35,096 fail2ban.transmitter [6770]: WARNING Command ['set', 'smtp-auth', 'addfailregex', 'sendmail.authentication failure \(-13\) SASL\(-13\):'] has failed. Received RegexException("No failure-id group in 'sendmail.authentication failure \(-13\) SASL\(-13\):'",)

2019-10-19 15:45:35,096 fail2ban [6770]: ERROR NOK: ("No failure-id group in 'sendmail.*authentication failure \(-13\) SASL\(-13\):'",)

2019-10-19 15:45:35,096 fail2ban.filter [6770]: INFO encoding: UTF-8

2019-10-19 15:45:35,096 fail2ban.filter [6770]: INFO maxRetry: 3

2019-10-19 15:45:35,096 fail2ban.filter [6770]: INFO findtime: 43200

2019-10-19 15:45:35,096 fail2ban.actions [6770]: INFO banTime: 86400 2019-10-19 15:45:35,099 fail2ban.jail [6770]: INFO Jail 'sshd' started 2019-10-19 15:45:35,103 fail2ban.jail [6770]: INFO Jail 'recidive' started 2019-10-19 15:45:35,104 fail2ban.filtersystemd [6770]: NOTICE Jail started without 'journalmatch' set. Jail regexs will be checked against all journal entries, which is not advised for performance reasons. 2019-10-19 15:45:35,109 fail2ban.jail [6770]: INFO Jail 'smtp-auth' started 2019-10-19 15:45:35,111 fail2ban.transmitter [6770]: WARNING Command ['add', 'sshd'] has failed. Received NameError("name 'noduplicates' is not defined",) 2019-10-19 15:45:35,160 fail2ban.transmitter [6770]: WARNING Command ['add', 'recidive'] has failed. Received NameError("name 'noduplicates' is not defined",) 2019-10-19 15:45:35,207 fail2ban.transmitter [6770]: WARNING Command ['add', 'smtp-auth'] has failed. Received NameError("name 'noduplicates' is not defined",)`

Here my smtp-auth.conf file

cat /etc/fail2ban/filter.d/smtp-auth.conf [Definition]

failregex = sendmail.*authentication failure: checkpass failed, relay=[]$

failregex = sendmail.authentication failure (-13) SASL(-13): authentication failure:.relay=.*[] ignoreregex =

gcoter commented 4 years ago

I can see some errors because fail2ban couldn't send its emails:

2019-10-16 23:38:52,323 fail2ban.action         [923]: ERROR   printf %b "Hi,\n
The jail sshd has been started successfully.\n
Regards,\n
Fail2Ban"|mail -s "[Fail2Ban] sshd: started on `uname -n`" <MY-GMAIL-ADDRESS> -- timed out after 60 seconds.
2019-10-16 23:38:52,425 fail2ban.action         [923]: ERROR   printf %b "Hi,\n
The jail sshd has been started successfully.\n
Regards,\n
Fail2Ban"|mail -s "[Fail2Ban] sshd: started on `uname -n`" <MY-GMAIL-ADDRESS> -- stdout: b''
2019-10-16 23:38:52,426 fail2ban.action         [923]: ERROR   printf %b "Hi,\n
The jail sshd has been started successfully.\n
Regards,\n
Fail2Ban"|mail -s "[Fail2Ban] sshd: started on `uname -n`" <MY-GMAIL-ADDRESS> -- stderr: b''
2019-10-16 23:38:52,427 fail2ban.action         [923]: ERROR   printf %b "Hi,\n
The jail sshd has been started successfully.\n
Regards,\n
Fail2Ban"|mail -s "[Fail2Ban] sshd: started on `uname -n`" <MY-GMAIL-ADDRESS> -- killed with SIGTERM (return code: -15)

Apart from that, the rest seems normal. I don't have a smtp-auth.conf file like you.

hellresistor commented 4 years ago

I don't have a smtp-auth.conf file like you.

it's possible share your smtp conf ?

gcoter commented 4 years ago

Do you mean the files in /etc/exim4/? I followed the instructions from this section: https://github.com/imthenachoman/How-To-Secure-A-Linux-Server#gmail-and-exim4-as-mta-with-implicit-tls

To be more precise, here are the files I modified:

...

.ifdef TLS_ON_CONNECT_PORTS tls_on_connect_ports = TLS_ON_CONNECT_PORTS .endif



Apart from these files, I didn't configure anything else in Exim4.
gcoter commented 4 years ago

Also, I configured UFW to allow traffic on port 465 as instructed in the tutorial.

hellresistor commented 4 years ago

I'm sorry my bad. i am talk about fail2ban config to exim/sendmail.conf ?

gcoter commented 4 years ago

Haha, no problem :) Did you mean this file?

/etc/fail2ban/action.d/sendmail.conf

# Fail2Ban configuration file
#
# Author: Cyril Jaquier
#
#

[INCLUDES]

before = sendmail-common.conf

[Definition]

# Option:  actionban
# Notes.:  command executed when banning an IP. Take care that the
#          command is executed with Fail2Ban user rights.
# Tags:    See jail.conf(5) man page
# Values:  CMD
#
actionban = printf %%b "Subject: [Fail2Ban] <name>: banned <ip> from `uname -n`
            Date: `LC_ALL=C date +"%%a, %%d %%h %%Y %%T %%z"`
            From: <sendername> <<sender>>
            To: <dest>\n
            Hi,\n
            The IP <ip> has just been banned by Fail2Ban after
            <failures> attempts against <name>.\n
            Regards,\n
            Fail2Ban" | /usr/sbin/sendmail -f <sender> <dest>

[Init]

# Default name of the chain
#
name = default

I think this is the default file. I didn't modify it myself.

gcoter commented 4 years ago

I also have this one (which is surely more relevant since I use exim).

/etc/fail2ban/filter.d/exim.conf

# Fail2Ban filter for exim
#
# This includes the rejection messages of exim. For spam and filter
# related bans use the exim-spam.conf
#

[INCLUDES]

# Read common prefixes. If any customizations available -- read them from
# exim-common.local
before = exim-common.conf

[Definition]

failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|Unrouteable address|all relevant MX records point to non-existent hosts)\s*$
            ^%(pid)s \w+ authenticator failed for (\S+ )?\(\S+\) \[<HOST>\](?::\d+)?(?: I=\[\S+\](:\d+)?)?: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
            ^%(pid)s %(host_info)sF=(?:<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: (?:relay not permitted|Sender verify failed|Unknown user)\s*$
            ^%(pid)s SMTP protocol synchronization error \([^)]*\): rejected (?:connection from|"\S+") %(host_info)s(?:next )?input=".*"\s*$
            ^%(pid)s SMTP call from \S+ %(host_info)sdropped: too many nonmail commands \(last was "\S+"\)\s*$
            ^%(pid)s SMTP protocol error in "AUTH \S*(?: \S*)?" %(host_info)sAUTH command used when not advertised\s*$
            ^%(pid)s no MAIL in SMTP connection from (?:\S* )?(?:\(\S*\) )?%(host_info)sD=\d+s(?: C=\S*)?\s*$
            ^%(pid)s \S+ SMTP connection from (?:\S* )?(?:\(\S*\) )?%(host_info)sclosed by DROP in ACL\s*$

ignoreregex = 

# DEV Notes:
# The %(host_info) defination contains a <HOST> match
#
# SMTP protocol synchronization error \([^)]*\)  <- This needs to be non-greedy
# to void capture beyond ")" to avoid a DoS Injection vulnerabilty as input= is
# user injectable data.
#
# Author: Cyril Jaquier
#         Daniel Black (rewrote with strong regexs)
#         Martin O'Neal (added additional regexs to detect authentication failures, protocol errors, and drops)
gcoter commented 4 years ago

Since it is possible that Google has blocked my server from authenticating (honestly I don't have other hypothesis at this point), I should probably setup my own SMTP server. Have you resources about doing this?

Today I tried to reconfigure Exim to use it as a Send-only Mail Server following this tutorial : https://www.linode.com/docs/email/exim/deploy-exim-as-a-send-only-mail-server-on-ubuntu-12-04/

But it seems that Google still refuses my mail:

2019-11-01 15:50:27 1iQYFu-0001ig-CM H=gmail-smtp-in.l.google.com [2a00:1450:400c:c09::1b] Network is unreachable
2019-11-01 15:50:27 1iQYFu-0001ig-CM H=gmail-smtp-in.l.google.com [74.125.140.27] Connection refused
2019-11-01 15:50:27 1iQYFu-0001ig-CM H=alt1.gmail-smtp-in.l.google.com [209.85.233.27] Connection refused
2019-11-01 15:50:27 1iQYFu-0001ig-CM H=alt2.gmail-smtp-in.l.google.com [172.217.194.27] Connection refused
2019-11-01 15:50:27 1iQYFu-0001ig-CM H=alt3.gmail-smtp-in.l.google.com [108.177.97.27] Connection refused
2019-11-01 15:50:27 1iQYFu-0001ig-CM H=alt4.gmail-smtp-in.l.google.com [2607:f8b0:400e:c00::1b] Network is unreachable

I am a bit desperate because, even when just sending a mail (not even using gmail's SMTP), it looks like my server is blocked. I just want my server to be able sending mails to my personal gmail address. Are there alternative solutions?

hellresistor commented 4 years ago

I think On this situation you need set the hosts file right way. something like this: `127.0.0.1 localhost

127.0.1.1 test.example.com test

xxx.xxx.xx.x test.example.com test

::1 localhost`

I think write right way

lgaida commented 4 years ago

I ran into the same issue while setting up a new server. PR #44 saved me, maybe you should check that out too and configure your /etc/exim4/passwd.client accordingly.

Btw: i'm using 2FA with an app-password

gcoter commented 4 years ago

Thank you for your suggestions, I will try them ASAP

imthenachoman commented 4 years ago

Hello all. Sorry I haven't replied. I'm caught up in the middle of some personal stuff and haven't had a chance to work on this. Hoping to get to it in the next few weeks!

hellresistor commented 4 years ago

Check new method ;) simple way SSMTP ;)Check Pull Requests ;)

Other thing about gmail. maybe you need Enable Less Secure APPs on google account!!

gcoter commented 4 years ago

I finally managed to make it work thanks to PR #44 and with 2FA. Thank you :)