search

nextcloud / mail

💌 Mail app for Nextcloud
https://apps.nextcloud.com/apps/mail
GNU Affero General Public License v3.0
841 stars 260 forks source link

INBOX not displayed since version 1.3.4 and in 1.4.0 - no login into dovecot - other folders displayed #3095

Closed pete-dawgg closed 3 years ago

pete-dawgg commented 4 years ago

already posted this in the nc-forum after update to 1.3.4 no emails in existing account were shown and trying to set up new account failed with "Imap error occurred" in webui. when trying to retrieve email with exisiting account dovecot.log states:

Info: Disconnected (no auth attempts in 0 secs): user=<>, rip=XXX lip=XXX, TLS handshaking: SSL_accept() failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca: SSL alert number 48, session=

Setting 'app.mail.verify-tls-peer' => false, in NC-DIR/config/config.php allowed to create new imap-account, but retrieving email is still not possible. With the above setting active the circle in the mail-app is spinning but dovecot shows no login-attempts when logging into nextcloud and opening the mail-app. nextcloud.log shows (with occ log:watch; XXX redacted):

Debug    mail               loading messages of folder <XXX=>  2020-05-11T10:15:43+00:00 
Debug    mail               loading messages of folder <XXX=> 2020-05-11T10:15:44+00:00
Debug    mail               account is up to date, skipping mailbox sync 2020-05-11T10:16:02+00:00
Debug    mail               account is up to date, skipping mailbox sync 2020-05-11T10:16:02+00:00 
Debug    mail               loading messages of folder <XXX=> 2020-05-11T10:16:03+00:00
Debug    mail               loading messages of folder <XXX=> 2020-05-11T10:16:18+00:00

and nothing else.

after removing the above setting from the config the previous ssl-error showed in dovecot again: ... TLS handshaking: SSL_accept() failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca: SSL alert number 48, ...

and nextcloud.log states:

Fatal    mail               OCA\Mail\Exception\ServiceException: IMAP errorError connecting to mail server. at apps/mail/lib/IMAP/MailboxSync.php line 88                                            

                              0. apps/mail/lib/Service/MailManager.php line 102                                                                                                                                                  
                                 OCA\Mail\IMAP\MailboxSync->sync(OCA\Mail\Account {})                                                                                                                                            
                              1. apps/mail/lib/Controller/PageController.php line 119                                                                                                                                            
                                 OCA\Mail\Service\MailManager->getFolders(OCA\Mail\Account {})                                                                                                                                   
                              2. lib/private/AppFramework/Http/Dispatcher.php line 170                                                                                                                                           
                                 OCA\Mail\Controller\PageController->index(                                                                                                                                                      

                                 )                                                                                                                                                                                               
                              3. lib/private/AppFramework/Http/Dispatcher.php line 99                                                                                                                                            
                                 OC\AppFramework\Http\Dispatcher->executeController(OCA\Mail\Controller\PageController {}, "index")                                                                                              
                              4. lib/private/AppFramework/App.php line 125                                                                                                                                                       
                                 OC\AppFramework\Http\Dispatcher->dispatch(OCA\Mail\Controller\PageController {}, "index")                                                                                                       
                              5. lib/private/AppFramework/Routing/RouteActionHandler.php line 47                                                                                                                                 
                                 OC\AppFramework\App::main("OCA\\Mail\\Controller\\PageController", "index", OC\AppFramework\DependencyInjection\DIContainer {}, {_route:"mail.page.index"})                                     
                              6. <<closure>>                                                                                                                                                                                     
                                 OC\AppFramework\Routing\RouteActionHandler->__invoke({_route:"mail.page.index"})                                                                                                                
                              7. lib/private/Route/Router.php line 299                                                                                                                                                           
                                 call_user_func(OC\AppFramework\Routing\RouteActionHandler {}, {_route:"mail.page.index"})                                                                                                       
                              8. lib/base.php line 1008                                                                                                                                                                          
                                 OC\Route\Router->match("\/apps\/mail\/")                                                                                                                                                        
                              9. index.php line 38                                                                                                                                                                               
                                 OC::handleRequest(                                                                                                                                                                              

                                 )

with the above setting in place again there are just no login attempts into dovecot.

Mail app version: 1.3.4 Mailserver or service: dovecot v 2.3.4.1

Server configuration

local dovecot (and nextcloud) installation on internet host Operating system: Debian Buster amd64 Web server: apache 2.4.38 Database: mariadb 10. PHP version: (e.g. 7.0) 7.3 Nextcloud Version: (see admin page, e.g. 13.0.2) 18.0.4

Client configuration

Browser: firefox all versions

this mail-setup causes no problems with oter software/servers.

ChristophWurst commented 4 years ago

with the above setting in place again there are just no login attempts into dovecot.

Please enable debug mode. Then, try to reproduce your issue and take another look at data/horde_imap.log. It should tell us why it does not connect.

pete-dawgg commented 4 years ago

Thany You! The horde_imap.log is here and also attached. It's only 113 lines - i logged into nc and started the mail app, waited until tail -f did not show any more lines being added and logged out again. The setting 'app.mail.verify-tls-peer' => false, was active when doing this.

horde_imap.log

ChristophWurst commented 4 years ago

The horde_imap.log is here and also attached.

Those look clean.

What I still do not fully grasp is that you need to disable to verify the TLS verification. So you can set up your account. And apparently it does fetch some data from IMAP. But then it fails with an SSL error? Are you sure that is not an issue with your setup, like changing IP addresses and getting connected to the wrong host? Do you use dynamic IPs?

pete-dawgg commented 4 years ago

It's exactly as descibed in the nc-forumpost mentioned above. The host is a VPS with a fixed IP and correct DNS-setup; mailsystem is fully functional. When i set up an account with TLS-verify enabled it just fails with an unspefic errmsg in the webui.

I did some more checking and testing and believe the error is not (really) caused by SSL but something else. Another user with the same mailserver-settings has no prolems at all; also (in the "defective" useraccounts the problem for nextcloud-mail seems only to exist with the "main" INBOX-folder. When opening other folders in nextcloud-mail their contents are displayed (eg SENT show all msgs including recently sent from other devices) and this is shown in the dovecot.log . When just "opening" thenextcloud-mail-app nothing shows in the dovecot-logs; same thing happens when opening the "main" inbox. The number of unread msgs is displayed correctly when using the three dots next to "Posteingang" Can you make any sense of this? nc-mail-red

pete-dawgg commented 4 years ago

I just updated to nextcloud 18.0.6 and the latest available version of the mail-app (1.4.0) For must users it is still not possible to display their INBOX-contents. I noticed that everything BUT the inbox works - mails can be sent, sent mails displayed, trash displayed and deleted etc., but everything related to the inbox and new emails just stays blank. i noticed that when "opening" the working "folders mentioned above there are connections logged in the dovecot log, but when "opening" the inbox(es) no login attempt is made to dovecotat all (caching issue?).

(Should I change the title of this bug since it seems to have nothing to do with SSL anymore?)

ChristophWurst commented 4 years ago

You can do that or start a fresh ticket. I'm still having a hard time following your issue descriptions. Not quite sure what is going on.

pete-dawgg commented 4 years ago

just changed the title. i just deleted one user's mail-account (inside mail-app) and re-added, but the proble persists in exactly the same way. is there any way to delete everything that is is cached in the context of the mail-app and this user?

ChristophWurst commented 4 years ago

if you create a new account the cache is created anew as well

pete-dawgg commented 4 years ago

Just to exclude any errors with the local mailserver i set up a new account using an external mailserver and the problem persists in exactly the same way. the contents of the inbox are not shown at all (and probably no imap-login-attempt is made since no horde.log is created at all); but when clicking on the three dots the message-summary of the account's inbox (YY unread of XX messages) is shown (and also a horde.log created that shows a summary of the imap-commands to the external mailserver. The sent messages are shown.

(This happens with and without the setting 'app.mail.verify-tls-peer' => false in config.php - though i guess it does not have anything to do with it.)

th3raid0r commented 4 years ago

I also get the same/similar behavior with Hydroxide Proton Mail bridge. I also have verify-tls-peer disabled. More notes here: https://github.com/nextcloud/mail/issues/3409#issuecomment-685058914

digititus commented 4 years ago

Same problem with Dovecot:

[mail] Error: OCA\Mail\Exception\ServiceException: IMAP errorError connecting to mail server. at <>

  1. /usr/share/nginx/nextcloud/apps/mail/lib/BackgroundJob/SyncJob.php line 91 OCA\Mail\IMAP\MailboxSync->sync(OCA\Mail\Account {}, true)
  2. /usr/share/nginx/nextcloud/lib/public/BackgroundJob/Job.php line 79 OCA\Mail\BackgroundJob\SyncJob->run({accountId: 3})
  3. /usr/share/nginx/nextcloud/lib/public/BackgroundJob/TimedJob.php line 61 OCP\BackgroundJob\Job->execute(OC\BackgroundJob\JobList {}, OC\Log {})
  4. /usr/share/nginx/nextcloud/cron.php line 148 OCP\BackgroundJob\TimedJob->execute(OC\BackgroundJob\JobList {}, OC\Log {})

GET /cron.php

[mail] Fatal: OCA\Mail\Exception\ServiceException: IMAP errorError connecting to mail server. at <>

  1. /usr/share/nginx/nextcloud/apps/mail/lib/Service/MailManager.php line 101 OCA\Mail\IMAP\MailboxSync->sync(OCA\Mail\Account {})
  2. /usr/share/nginx/nextcloud/apps/mail/lib/Controller/PageController.php line 119 OCA\Mail\Service\MailManager->getFolders(OCA\Mail\Account {})
  3. /usr/share/nginx/nextcloud/lib/private/AppFramework/Http/Dispatcher.php line 170 OCA\Mail\Controller\PageController->index()
  4. /usr/share/nginx/nextcloud/lib/private/AppFramework/Http/Dispatcher.php line 99 OC\AppFramework\Http\Dispatcher->executeController(OCA\Mail\Controller\PageController {}, "index")
  5. /usr/share/nginx/nextcloud/lib/private/AppFramework/App.php line 125 OC\AppFramework\Http\Dispatcher->dispatch(OCA\Mail\Controller\PageController {}, "index")
  6. /usr/share/nginx/nextcloud/lib/private/AppFramework/Routing/RouteActionHandler.php line 47 OC\AppFramework\App::main("OCA\Mail\Controller\PageController", "index", OC\AppFramework\ ... {}, {_route: "mail.page.index"})
  7. <> OC\AppFramework\Routing\RouteActionHandler->__invoke({_route: "mail.page.index"})
  8. /usr/share/nginx/nextcloud/lib/private/Route/Router.php line 299 call_user_func(OC\AppFramework\ ... {}, {_route: "mail.page.index"})
  9. /usr/share/nginx/nextcloud/lib/base.php line 1013 OC\Route\Router->match("/apps/mail/")
  10. /usr/share/nginx/nextcloud/index.php line 38 OC::handleRequest()

GET /apps/mail/

ChristophWurst commented 4 years ago

@digititus see https://github.com/nextcloud/mail/issues/3095#issuecomment-626663567. We might know more if you sent us those logs.

duesseldorferjung commented 4 years ago

I am facing a similar issue. Could you please provide me some guidance how to get the logs. If I look at my logs I don't see anything for mail app so far.

pete-dawgg commented 4 years ago

Could you please provide me some guidance how to get the logs.

First answer to my initial post: enable debug log and then data/horde_imap.log.

pete-dawgg commented 3 years ago

sorry, closed by accident. issue remains exactly the same.

ChristophWurst commented 3 years ago

Please run a manual account sync and post the output here: https://github.com/nextcloud/mail/blob/master/doc/admin.md#manual-account-synchronization-and-threading

redact mailbox names if they contain any PII.

pete-dawgg commented 3 years ago

THX for your help! i ran the command and lots of times nothing happened (log in and log out seen in dovecot.log or "Account already being synced") but some meaningful erros were thrown. here ist the slightly redacted (XXX) output:

sudo -u www-data /usr/bin/php /var/www/nc/occ mail:account:sync  XXX

In ImapToDbSynchronizer.php line 213:

Sync failed for XXX:INBOX: An exception occurred while executing 'INSERT INTO `oc_mail_messages` (`uid`, `message_id`, `mailbox_id`, `subject`, `sent_at`, `flag_answered`, `flag_deleted`, `flag_draft`, `flag_flagged`, `flag_seen`, `flag_forwarded`, `flag_junk`, `flag_notjunk`, `flag_important`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [32555, "<0A990F81-5907FC336D4-016A03719B0C-000000000492C76B@XXXXXXXX>", 86, "\ud83d\ude9a XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXX XXX...", 1556608413, false, false, false, false, true, false, false, false, false]:                               

SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x9A\x9A A...' for column `nextcloud`.`oc_mail_messages`.`subject` at row 1

In AbstractMySQLDriver.php line 106:
An exception occurred while executing 'INSERT INTO `oc_mail_messages` (`uid`, `message_id`, `mailbox_id`, `subject`, `sent_at`, `flag_answered`, `flag_deleted`, `flag_draft`, `flag_flagged`, `flag_seen`, `flag_forwarded`, `flag_junk`, `flag_notjunk`, `flag_important`) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)' with params [32555, "<0A990F81-5907FC336D4-016A03719B0C-000000000492C76B@XXXXXXXX>", 86, "\ud83d\ue9a XXXXXXXXXXXXXXXXXXXXXX\u00e4XXXXXXX XXXXXXXXXXXXXXXXXXXX...", 1556608413, false, false, false, false, true, false, false, false, false]:                                                                                                                                           
SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x9A\x9A A...' for column `nextcloud`.`oc_mail_messages`.`subject` at row 1

In PDOStatement.php line 119:
SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x9A\x9A A...' for column `nextcloud`.`oc_mail_messages`.`s ubject` at row 1

In PDOStatement.php line 117:
SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x9A\x9A A...' for column `nextcloud`.`oc_mail_messages`.`subject` at row 1

mail:account:sync [-f|--force] [--] <account-id>
pete-dawgg commented 3 years ago

some additional info: i manually deleted the mail with the string (?) causing this error from the dovecot-files, but now the exact same error is caused by another mail in:

In ImapToDbSynchronizer.php line 213: In AbstractMySQLDriver.php line 106: In PDOStatement.php line 119: In PDOStatement.php line 117:

this error is thrown: SQLSTATE[22007]: Invalid datetime format: 1366 Incorrect string value: '\xF0\x9F\x8E\x81 R...' for columnnextcloud.oc_mail_messages.subjectat row 1

Hope this helps a little.

pete-dawgg commented 3 years ago

PROGRESS (some)! i manually deleted some of the error-causing emails in my personal (test)account and did a dovecot expunge (since these mails had been put in the trash some time ago) and now emails are displayed again in the nextcloud webmail-if. Haven't checked this with any users yet. However, manual sync with occ still causes the same errors, and i cannot do the above for all users and grep&delete in their dovecot-files manually.

ChristophWurst commented 3 years ago

https://github.com/nextcloud/mail/pull/4442 should fix or at least help with this issue. we also backported this to the 1.8.x series.