search

revpriest / listman

An email-list manager for NextCloud
GNU General Public License v3.0
18 stars 2 forks source link

smtp connection impossible to make it working #18

Closed francoisPE closed 1 year ago

francoisPE commented 2 years ago

Hello, I run nextcloud 23.0.5 with listman 20.1.1 on ubuntu server. My mails are configured in postfix with ispconfig.

Nextcloud is sending emails and my mail server is reachable by telnet mydom.tls 587

I can't succeed in sending mail with listman. I have big difficulties to troubleshoot.

I open vendor/phpmailer/phpmailer/src/PHPMailer.ph I set $SMTPDebug = 1 and $Debugoutput = 'echo' and obtain

2022-06-20 16:55:06     SMTP ERROR: Failed to connect to server:  (0)
2022-06-20 16:55:06     SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting

Quite short to troubleshoot !

I try to modify $SMTPSecure = 'tls'; to force tls. No success.

What is very strange : for all these tests, I see no connection on my mail server. No connection attemp ! Same mood : for at least 5 tests, I see in browser "latest Warn" box

 2022-06-20 17:00:05: Not sending to myaddress@dom.fr today, already passed sending limits. Will do it later

Mail limit is at 500... I made 10+ tests... I try to see if number of mail sent is in mysql : seems not to be !

Any idea ?

revpriest commented 2 years ago

Not much of an idea really I'm afraid. The PHPMailer isn't mine, it's just vendor code. I know a bit of SMTP protocol but nothing at all about how to SSL over it other than "use the library".

All my code really does is pack a message for https://github.com/PHPMailer/PHPMailer

Does it work if you use a different mail-host? Or one that isn't SSL?

Sounds like you've already done more debugging than I would have known how to do to it.

It says here ( https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting#check-you-have-the-openssl-extension ) that your PHP has to have been compiled with openssl built in, but I guess it must have been or else it wouldn't send for nextcloud? Unless nextcloud has it's own SSL implementation.

Could it be that?

francoisPE commented 2 years ago

I am not sure it is only an issue with PHPMailer.

apps shows that image

last Warn box says

2022-06-21 11:30:07: Error sending to mail@dom.fr - SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting

My nextcloud log says

/var/www/nc.dom.fr/private/nc.log:{"reqId":"UrMlBgy7d3NfvEOIdXlZ","level":3,"time":"2022-06-21T13:30:07+02:00","remoteAddr":"","user":"--","app":"listman","method":"","url":"--","message":"Error sending to mail@dom.fr - SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting","userAgent":"--","version":"23.0.5.1","exception":{"Exception":"PHPMailer\\PHPMailer\\Exception","Message":"SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting","Code":2,"Trace":[{"file":"/myppath/web/apps/listman/vendor/phpmailer/phpmailer/src/PHPMailer.php","line":1648,"function":"smtpSend","class":"PHPMailer\\PHPMailer\\PHPMailer","type":"->"},{"file":"/mypath/web/apps/listman/vendor/phpmailer/phpmailer/src/PHPMailer.php","line":1486,"function":"postSend","class":"PHPMailer\\PHPMailer\\PHPMailer","type":"->"},{"file":"/mypath/web/apps/listman/lib/Service/ListmanService.php","line":1063,"function":"send","class":"PHPMailer\\PHPMailer\\PHPMailer","type":"->"},{"file":"/mypath/web/apps/listman/lib/Service/ListmanService.php","line":1118,"function":"sendEmailToMember","class":"OCA\\Listman\\Service\\ListmanService","type":"->"},{"file":"/mypah/web/apps/listman/lib/Cron/ListmanSend.php","line":28,"function":"runCron","class":"OCA\\Listman\\Service\\ListmanService","type":"->"},{"file":"/mypath/web/lib/public/BackgroundJob/Job.php","line":79,"function":"run","class":"OCA\\Listman\\Cron\\ListmanSend","type":"->"},{"file":"/mypath/web/cron.php","line":151,"function":"execute","class":"OCP\\BackgroundJob\\Job","type":"->"}],"File":"/mypath/web/apps/listman/vendor/phpmailer/phpmailer/src/PHPMailer.php","Line":1971,"CustomMessage":"Error sending to mail@dom.fr - SMTP connect() failed. https://github.com/PHPMailer/PHPMailer/wiki/Troubleshooting"}}

But... My mail server shows, which is not really an auth issue !

/var/log/mail.log:Jun 21 13:30:07 ns1 postfix/submission/smtpd[2966806]: connect from s1.dom.fr[ipv4]
/var/log/mail.log:Jun 21 13:30:07 ns1 postfix/submission/smtpd[2966806]: lost connection after CONNECT from s1.dom.fr[ipv4]
/var/log/mail.log:Jun 21 13:30:07 ns1 postfix/submission/smtpd[2966806]: disconnect from s1.dom.fr[ipv4] commands=0/0
/

I dig !

francoisPE commented 2 years ago

Hello, Can you help me make visible servers 'helo' exchanges...

revpriest commented 2 years ago

You are better at collecting this data than me. I like it.

But I still dunno what could be wrong.

Sounds to me likely there isn't a helo to show, that the connection is made but the SSL fails to negotiate.

So I guess it might be about what cyphers are allowed where?

Unfortunately I also know next to nothing about how SSL negotiates cyphers.

You seem wiser in these things than me, perhaps there's a way to view which cyphers are accepted by both the SSL in php and the SSL in your mail server?

I dunno if SSL keeps any logs of why it drops connections?

revpriest commented 2 years ago

Looks like you've already tried

$mail->SMTPSecure = PHPMailer::ENCRYPTION_STARTTLS;

That's the only "which type of cypher" setting there is that I know of.

If I could get a test account on your server I could spend a bit of time testing if I can get my setup to connect to it.

But not till early next month really.

revpriest commented 2 years ago

I will try and add an option to change the mailer SMTPSecure setting between ENCRYPTION_SMTPS and ENCRYPTION_STARTTLS when I get a chance, but sounds like you've already tried both alternatives in your case.

francoisPE commented 2 years ago

I find out in log exchanges : 

Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 220 mail1.dom.fr ESMTP Postfix (Ubuntu)
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]: ????
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad UTF-8 syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]: ?$??ߖ�??ҪK???;`?Ć???H{52?
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad UTF-8 syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]:
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad UTF-8 syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]:
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]:
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad syntax

I modify CHARSET_ISO88591 into UTF8 see below

# grep CHARSET vendor/phpmailer/phpmailer/src/PHPMailer.php                       
    const CHARSET_ASCII = 'us-ascii';
    const CHARSET_ISO88591 = 'iso-8859-1';
    const CHARSET_UTF8 = 'utf-8';
    public $CharSet = self::CHARSET_UTF8;
    public static function parseAddresses($addrstr, $useimap = true, $charset = self::CHARSET_UTF8)
                $domain = mb_convert_encoding($domain, self::CHARSET_UTF8, $this->CharSet);
                $is_utf8 = static::CHARSET_UTF8 === strtolower($this->CharSet);
                $bodyCharSet = static::CHARSET_ASCII;
                $altBodyCharSet = static::CHARSET_ASCII;
                $charset = static::CHARSET_ASCII;

But now

2022-06-21 21:40:16: Not sending to mail@dom.fr today, already passed sending limits. Will do it later

How this quota is calculated ?

francoisPE commented 2 years ago

I increased quota. Now 

Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 220 mail1.dom.fr ESMTP Postfix (Ubuntu)
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]: ????
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad UTF-8 syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]: 
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad UTF-8 syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]:
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad UTF-8 syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]:
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad syntax
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: < s1.dom.fr[ipv4]:
Jun 21 22:45:16 mail1 postfix/submission/smtpd[3054602]: > s1.dom.fr[ipv4]: 500 5.5.2 Error: bad syntax
francoisPE commented 2 years ago

It seems to be my nextcloud server which is not able to say something clear: ???? instead...

revpriest commented 2 years ago

You're doing great work trying to track it down, but I remain stumped. Character encoding or cipher selection still seem to be main things to look at. A broken cypher or wrong character encoding are going to produce nonsense in the stream like that. Afraid I know nothing about them and will also be away for the next week or so.

Good luck! Do let me know if you fix it.

francoisPE commented 2 years ago

You were clearly right My issue is solved by ; $mail->SMTPSecure = PHPMailer::ENCRYPTION_STARTTLS; in lib/Service/ListmanService.php My solution is not compatible with autodetection...

If you can put a toggle somewhere it would be very nice !

Thanks a lot for your help !

revpriest commented 2 years ago

Great, glad we found a solution. I'll add the option and push a new version when I get some time. Maybe this weekend.

revpriest commented 1 year ago

Pushed a new version to the repo and the app-store with an option to select tls/ssl for the mailer plugin. Hopefully that's good for fixing this.

francoisPE commented 1 year ago

I tested : it works nice. Thanks a lot for all what you did to help me !