Z-Push can't connect to local IMAP server

[axd]

Hi, I have a fresh installation of MiAB, buy I'm facing a problem trying to use an exchange account, I just configured this account but the email never load, not even the folders, nothing, here's an extaction of logs:

09/01/2017 00:01:28 [ 1357] [ERROR] [user@domain.com] BackendIMAP->Logon(): can't connect as user 'user@domain.com' on '{127.0.0.1:993/imap/ssl/norsh/novalidate-cert}': [CLOSED] IMAP connection broken (server response)

09/01/2017 00:01:35 [ 1356] [ERROR] [user@domain.com] BackendIMAP->Logon(): can't connect as user 'user@domain.com' on '{127.0.0.1:993/imap/ssl/norsh/novalidate-cert}': [CLOSED] IMAP connection broken (server response)

09/01/2017 00:01:42 [ 1357] [ERROR] [user@domain.com] BackendIMAP->Logon(): can't connect as user 'user@domain.com' on '{127.0.0.1:993/imap/ssl/norsh/novalidate-cert}': [CLOSED] IMAP connection broken (server response)

09/01/2017 00:02:00 [ 1357] [ERROR] [user@domain.com] BackendIMAP->Logon(): can't connect as user 'user@domain.com' on '{127.0.0.1:993/imap/ssl/norsh/novalidate-cert}': [CLOSED] IMAP connection broken (server response)

09/01/2017 00:02:28 [ 1356] [ERROR] [user@domain.com] BackendCardDAV->Logon(): User 'user@domain.com' failed to authenticate on 'https://127.0.0.1:443/carddav/addressbooks/user@domain.com/': 

09/01/2017 00:02:30 [ 2162] [ERROR] [user@domain.com] BackendCardDAV->Logon(): User 'user@domain.com' failed to authenticate on 'https://127.0.0.1:443/carddav/addressbooks/user@domain.com/':

Normal IMAP connections are working well, I can login in RC and check my email, also if I configure a normal IMAP account in my client, it works.

I look in a solved issue but those changes was already on my config files: https://github.com/mail-in-a-box/mailinabox/issues/264

I'm using the latest release.

Thanks.

[sa7lej]

Hi!

I had the same exact problem. My solution was to created a contact in owncloud for the account that i was connecting with and created a app password. I didn't use the app-password (i deleted it right away) for anything so i'm guessing that a contact was enough.

My theory (without actually knowing that much about z-push) is that one must create a contact to propagate /carddav/addressbooks/user@domain.com/ with data so that z-push have something to connect to.

[yodax]

Thanks for figuring this out.

I don't use z-push anymore (to many encoding issues), but when I did, I didn't have any contacts. Could this be related to: https://github.com/mail-in-a-box/mailinabox/pull/940 ? I don't really see how, since that change is about mail and not contacts.

[purplevegetable]

I'm pretty sure this thread relates to this issue too.

https://discourse.mailinabox.email/t/exchange-keeps-saying-incorrect-password/1765/6

[axd]

@yodax Z-Push not only serve contacts, it involves everything (email, contacts & calendar) so it's probable that any change in the connection config could lead in an issue.

@azzeman My theory is that Z-Push now depends on the creation of the user in Roundcube I'm supposing this because of this line on conf/zpush/backend_imap.php:

+define('IMAP_FROM_SQL_DSN', 'sqlite:STORAGE_ROOT/mail/roundcube/roundcube.sqlite');

@purplevegetable I did that, but was not successful at first time, I did it later and it works, so this is unstable for now.

[yodax]

Does this still occur with Nextcloud 12 and the latest z-push? I tried reproducing it again but I couldn't get it to fail. I added a new user and connected straight though z-push. No webmail/nextcloud logins. Everything worked as expected. (I could reproduce it a while ago)

[axd]

@yodax Let me check it out, I have to update first.

[jbandholz]

I can confirm this problem exists on 0.26b. I had a working configuration and then upgraded to 0.26 and these login problems started occurring.

from the z-push log: 02/02/2018 20:55:37 [ 7535] [WARN] [user@domain.com] BackendCalDAV->Logon(): User 'user@domain.com' is not authenticated on CalDAV 'https://127.0.0.1:443/caldav/calendars/user@domian.com/' 02/02/2018 20:55:37 [ 7535] [FATAL] [user@domain.com] Exception: (AuthenticationRequiredException) - Access denied. Username or password incorrect

also checked the nginx log and found 2018/02/02 20:55:37 [error] 3775#0: 3447 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: box.domain.com, request: "OPTIONS /caldav/calendars/user@domain.com/ HTTP/1.1 ", upstream: "https://127.0.0.1:443/cloud/remote.php/caldav/calendars/user@domain.com/", host: "127.0.0.1" 2018/02/02 20:55:37 [error] 3775#0: 3106 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 127.0.0.1, server: box.domain.com, request: "PROPFIND /caldav/calendars/user@domain.com/personal / HTTP/1.1", upstream: "https://127.0.0.1:443/cloud/remote.php/caldav/calendars/user@domain.com/personal/", host: "127.0.0.1"

update: I'm also seeing z-push errors like this: 02/02/2018 21:32:10 [21245] [ERROR] [user@domain.com] LoopDetection->ProcessLoopDetectionPreviousConnectionFailed(): Command 'FolderSync' at 02/02/2018 21:21:25 with pid '16045' terminated unexpectedly or is still running. 02/02/2018 21:32:10 [21245] [ERROR] [user@domain.com] Please check your logs for this PID and errors like PHP-Fatals or Apache segmentation faults and report your results to the Z-Push dev team.

[JoshData]

Is this problem still occurring?

[jbandholz]

Immediately after upgrading to v0.27, I was getting the same login failures continuously on all my android devices/accounts (z-push only; imap worked fine if a little bit slow; multiple server reboots did not resolve). I tried deleting accounts from the phones and re-syncing, and that completely failed. The phones timed-out communicating with the server (z-push only; adding account as IMAP worked). This led me to believe I was experiencing some sort of application load problem or hangup. I gave up debugging for the night, and by next morning, everything was working normally.

I still see this problem occasionally; at least once per week. The problem seems to come in bursts. One day, I will get Android login failures multiple times in one day and then will be clean the rest of the week.

The new batch reporting has given me some clues to try next. I'm seeing what I believe to be excessive logins for some of my accounts (see below, the top 3 accounts use z-push on multiple devices). I'll try to bring down the top offender and see if the problems continue.

User logins per hour ════════════════════

                                imap smtp mana │ timespan

─────────────────────────────────────────────────┼─────────── xxx1@domain.com 5223.7 0.0 0.0 │ 7.0 days xxx2@domain.com 1401.9 0.0 0.0 │ 7.0 days xxx3@domain.com 1066.0 0.0 0.0 │ 7.0 days xxx4@domain.com 115.8 0.0 0.0 │ 7.0 days xxx5@domain.com 95.3 0.1 0.0 │ 7.0 days ─────────────────────────────────────────────────┼─────────── Totals: 7902.7 0.1 0.0 │ 6.96 days ┬ totals by time of day: │ hour 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 │ imap 51305 28822 31455 33955 45269 44550 47466 44930 33975 54860 47477 53936 50267 52567 46181 57367 70969 85368 72859 79190 60941 78208 75992 77446 │ smtp 0 0 0 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 1 0 2 3 0 0 │ mana 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 2 0 0 0 0 0 └────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────