search

salesagility / SuiteCRM

SuiteCRM - Open source CRM for the world
https://www.suitecrm.com
GNU Affero General Public License v3.0
4.58k stars 2.11k forks source link

Gmail IMAP incoming mail #7186

Open haneef-faisal opened 5 years ago

haneef-faisal commented 5 years ago

Environment SuiteCRM Version 7.11.3 Sugar Version 6.5.25 (Build 344) Ubuntu 16.04.6 LTS Apache/2.4.18 PHP 7.3.3-1+ubuntu16.04.1+deb.sury.org+1 (cli) (built: Mar 7 2019 20:31:26) ( NTS ) Copyright (c) 1997-2018 The PHP Group Zend Engine v3.3.3, Copyright (c) 1998-2018 Zend Technologies with Zend OPcache v7.3.3-1+ubuntu16.04.1+deb.sury.org+1, Copyright (c) 1999-2018, by Zend Technologies MySQL 5.7.25 AWS EC2 Server; running only 1 SuiteCRM instance

The inbound emails were working till 2 days back; but now they won't import the emails. I am using GMail Apps IDs for Bounce Handling; currently there are 8 IDs with unread emails but my SuiteCRM instance won't process them.

I have already tried deleting all Inbound Emails (Bounce Handling) and creating only 1 account. On the SuiteCRM UI; the connection test is successful; however when I tail the suitecrm logs at debug level; I see the following:

Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert\/secure}INBOX: invalid remote specification" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert\/secure}INBOX: invalid remote specification" Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert\/secure}INBOX: invalid remote specification" Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert\/secure}INBOX: invalid remote specification" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert}INBOX: invalid remote specification" Thu Apr 11 18:24:29 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert}INBOX: invalid remote specification" Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert}INBOX: invalid remote specification" Thu Apr 11 18:24:29 2019 [5705][1][WARN] An Imap error detected: "Can't open mailbox {imap.gmail.com:993\/service=imap\/ssl\/tls\/validate-cert}INBOX: invalid remote specification" Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to set a non valid resource az stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][WARN] An Imap error detected: "IMAP open error" Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][WARN] An Imap error detected: "Can't do secure authentication with this server" Thu Apr 11 18:24:30 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:30 2019 [5705][1][WARN] An Imap error detected: "Can't do secure authentication with this server" Thu Apr 11 18:24:30 2019 [5705][1][WARN] An Imap error detected: "Can't do secure authentication with this server" Thu Apr 11 18:24:30 2019 [5705][1][WARN] An Imap error detected: "Can't do secure authentication with this server" Thu Apr 11 18:24:31 2019 [5705][1][WARN] Requested folder is not defined Thu Apr 11 18:24:31 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:31 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:31 2019 [5705][1][ERROR] ImapHandler trying to use a non valid resource stream. Thu Apr 11 18:24:31 2019 [5705][1][WARN] Connection is not a valid resource.

On Gmail Side I have already checked for:

Assuming that GMail might be blocking my server to connect for security reasons; I used the commands at https://tewarid.github.io/2011/05/10/access-imap-server-from-the-command-line-using-openssl.html to connect to the GMail account and was able to successfully connect. Herein is the log details:

username@ip-123-123-123-123:/var/www/html$ openssl s_client -crlf -connect imap.gmail.com:993 CONNECTED(00000003) depth=0 OU = "No SNI provided; please fix your client.", CN = invalid2.invalid verify error:num=18:self signed certificate verify return:1 depth=0 OU = "No SNI provided; please fix your client.", CN = invalid2.invalid verify return:1

Certificate chain 0 s:/OU=No SNI provided; please fix your client./CN=invalid2.invalid i:/OU=No SNI provided; please fix your client./CN=invalid2.invalid

Server certificate -----BEGIN CERTIFICATE----- MIIDfDCCAmSgAwIBAgIJAJB2iRjpM5OgMA0GCSqGSIb3DQEBCwUAME4xMTAvBgNV BAsMKE5vIFNOSSBwcm92aWRlZDsgcGxlYXNlIGZpeCB5b3VyIGNsaWVudC4xGTAX BgNVBAMTEGludmFsaWQyLmludmFsaWQwHhcNMTUwMTAxMDAwMDAwWhcNMzAwMTAx MDAwMDAwWjBOMTEwLwYDVQQLDChObyBTTkkgcHJvdmlkZWQ7IHBsZWFzZSBmaXgg eW91ciBjbGllbnQuMRkwFwYDVQQDExBpbnZhbGlkMi5pbnZhbGlkMIIBIjANBgkq hkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzWJP5cMThJgMBeTvRKKl7N6ZcZAbKDVA tNBNnRhIgSitXxCzKtt9rp2RHkLn76oZjdNO25EPp+QgMiWU/rkkB00Y18Oahw5f i8s+K9dRv6i+gSOiv2jlIeW/S0hOswUUDH0JXFkEPKILzpl5ML7wdp5kt93vHxa7 HswOtAxEz2WtxMdezm/3CgO3sls20wl3W03iI+kCt7HyvhGy2aRPLhJfeABpQr0U ku3q6mtomy2cgFawekN/X/aH8KknX799MPcuWutM2q88mtUEBsuZmy2nsjK9J7/y hhCRDzOV/yY8c5+l/u/rWuwwkZ2lgzGp4xBBfhXdr6+m9kmwWCUm9QIDAQABo10w WzAOBgNVHQ8BAf8EBAMCAqQwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC MA8GA1UdEwEB/wQFMAMBAf8wGQYDVR0OBBIEELsPOJZvPr5PK0bQQWrUrLUwDQYJ KoZIhvcNAQELBQADggEBALnZ4lRc9WHtafO4Y+0DWp4qgSdaGygzS/wtcRP+S2V+ HFOCeYDmeZ9qs0WpNlrtyeBKzBH8hOt9y8aUbZBw2M1F2Mi23Q+dhAEUfQCOKbIT tunBuVfDTTbAHUuNl/eyr78v8Egi133z7zVgydVG1KA0AOSCB+B65glbpx+xMCpg ZLux9THydwg3tPo/LfYbRCof+Mb8I3ZCY9O6FfZGjuxJn+0ux3SDora3NX/FmJ+i kTCTsMtIFWhH3hoyYAamOOuITpPZHD7yP0lfbuncGDEqAQu2YWbYxRixfq2VSxgv gWbFcmkgBLYpE8iDWT3Kdluo1+6PHaDaLg2SacOY6Go= -----END CERTIFICATE----- subject=/OU=No SNI provided; please fix your client./CN=invalid2.invalid issuer=/OU=No SNI provided; please fix your client./CN=invalid2.invalid

No client certificate CA names sent Peer signing digest: SHA256 Server Temp Key: X25519, 253 bits

SSL handshake has read 1386 bytes and written 373 bytes Verification error: self signed certificate

New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384 Server public key is 2048 bit Secure Renegotiation IS NOT supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.3 Cipher : TLS_AES_256_GCM_SHA384 Session-ID: Session-ID-ctx: Resumption PSK: 000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 PSK identity: None PSK identity hint: None SRP username: None Start Time: 1555020226 Timeout : 7200 (sec) Verify return code: 18 (self signed certificate) Extended master secret: no Max Early Data: 0

Any assistance on this would be highly appreciated.

c0cl3c commented 5 years ago

Same issue, but with Exchange Online. Went through the same online tests. Can connect directly from CLI via the server. Same error in suitecrm.log.

Weird thing is that I have a Zuver server that has basic IMAP support. I created a subdomain for email and set this up as a Group email address and it does come through. But the Case Creation template is broken. Does not respond with the Case number, etc.

Whats really weird is the case creation response is coming from the System email setup in SCRM, not the email details entered for the Group Mailbox or from AOP.

greenlanegreb commented 5 years ago

and still not fixed on a clean install of 7.11.7 (or fixed on today's upgrade to 7.11.8).

timo12357 commented 4 years ago

Same problem on 7.10.22 LTS

ebogaard commented 3 years ago

This is actually due to an issue with php-imap in combination with TLS 1.3, which is enabled by openssl 1.1.1x. For that TLS-version, SNI isn't enabled, which leads to the issues shown with imap.google.com, as shown in the start post. See: https://bugs.launchpad.net/ubuntu/+source/php-imap/+bug/1834340

As I understand, php-imap is based on an ancient and unsupported library, see: https://forum.remirepo.net/viewtopic.php?pid=11856 This is the reason modern distro's don't package php-imap anymore.

So the only proper solution is to implement a different imap library, as is proposed in #8942 as well.