Search Bug at SoGo - Related to Mailcow backend -> dovecot?

kcastner commented 2 years ago

Prior to placing the issue, please check following: (fill out each checkbox with an X once done)

[X ] I understand that not following or deleting the below instructions will result in immediate closure and/or deletion of my issue.
[X ] I have understood that this bug report is dedicated for bugs, and not for support-related inquiries.
[X ] I have understood that answers are voluntary and community-driven, and not commercial support.
[X ] I have verified that my issue has not been already answered in the past. I also checked previous issues.

Summary

A search via SoGo fails 8 times out of 10 with the following error message in the bottom right UI:
An error occurred while communicating with the email server.

The search is aborted according to the log because the "mailbox" does not exist.
If I start the search again, it finds a different number of mails.

Sometimes the search runs correctly the first time.

Logs

Feb 23 15:09:06 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderRewe/view HTTP/1.0" 200 665/131 0.200 - - 0 - 14
Feb 23 15:09:06 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderShelly/view HTTP/1.0" 200 456/131 0.196 - - -24K - 14
Feb 23 15:09:06 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderShopify/view HTTP/1.0" 200 80/131 0.180 - - 0 - 14
Feb 23 15:09:07 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderSipgate/view HTTP/1.0" 200 82/131 0.285 - - 0 - 14
Feb 23 15:09:07 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderSipgate-calls/view HTTP/1.0" 200 82/131 0.399 - - 0 - 14
Feb 23 15:09:07 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderSlitech/view HTTP/1.0" 200 80/131 0.165 - - 0 - 14
Feb 23 15:09:08 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderSmartdeploy/view HTTP/1.0" 200 82/131 0.207 - - 0 - 14
Feb 23 15:09:08 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderSolaris/view HTTP/1.0" 200 80/131 0.172 - - 0 - 14
Feb 23 15:09:08 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderSpotify/view HTTP/1.0" 200 420/131 0.222 - - 0 - 14
Feb 23 15:09:08 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderSteam/view HTTP/1.0" 200 81/131 0.190 - - 0 - 14
Feb 23 15:09:09 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderTelekom/view HTTP/1.0" 200 82/131 0.346 - - 0 - 14
Feb 23 15:09:09 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderTest/view HTTP/1.0" 200 80/131 0.162 - - 0 - 14
Feb 23 15:09:09 c1afbc4d8391 sogod [62]: [ERROR] <0x0x556e227e5f10[NGImap4Connection]> could not select URL: imap://exampleuser%40example.tld@172.22.1.250/INBOX/Tevi/?tls=NO&tlsVerifyMode=none: {RawResponse = "{ResponseResult = {description = \"Mailbox doesn't exist: INBOXTevi (0.002 + 0.000 + 0.001 secs).\"; result = no; tagId = 4; }; }"; reason = "Mailbox doesn't exist: INBOXTevi (0.002 + 0.000 + 0.001 secs)."; result = 0; }
Feb 23 15:09:09 c1afbc4d8391 sogod [62]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderTevi/view HTTP/1.0" 500 72/131 0.121 - - 0 - 14
Feb 23 15:09:12 c1afbc4d8391 sogod [66]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/changes HTTP/1.0" 200 25/149 0.098 - - 0 - 13
Feb 23 15:09:14 c1afbc4d8391 sogod [66]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/view HTTP/1.0" 200 682/131 0.695 - - 0 - 14
Feb 23 15:09:15 c1afbc4d8391 sogod [66]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folder_1und1/view HTTP/1.0" 200 83/131 0.603 - - 0 - 14
Feb 23 15:09:15 c1afbc4d8391 sogod [66]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderBetreut/view HTTP/1.0" 200 82/131 0.346 - - 0 - 14
Feb 23 15:09:15 c1afbc4d8391 sogod [66]: CLIENT_IP_REPLACED "POST /SOGo/so/exampleuser@example.tld/Mail/0/folderINBOX/folderBf3/view HTTP/1.0" 200 80/131 0.167 - - 0 - 14
Feb 23 15:09:15 c1afbc4d8391 sogod [66]: [ERROR] <0x0x556e21bdec40[NGImap4Connection]> could not select URL: imap://exampleuser%40example.tld@172.22.1.250/INBOX/Bitwarden/?tls=NO&tlsVerifyMode=none: {RawResponse = "{ResponseResult = {description = \"Mailbox doesn't exist: INBOXBitwarden (0.002 + 0.000 + 0.001 secs).\"; result = no; tagId = 4; }; }"; reason = "Mailbox doesn't exist: INBOXBitwarden (0.002 + 0.000 + 0.001 secs)."; result = 0; }

Reproduction

Login to SoGo to an Account +500MB Mailbox size..
on the left side, you can find your mail address
klick at 3 dots on the Right side
search
at criterion klick Content
enter some words.. in my case "PayPal"
search Folder should be your email address
search!

System information

Question	Answer
My operating system	Debian 11 - bullseye
Is Apparmor, SELinux or similar active?	dont know how to check
Virtualization technlogy (KVM, VMware, Xen, etc - LXC and OpenVZ are not supported	KVM
Server/VM specifications (Memory, CPU Cores)	8GB RAM, 8GB SWAP; 4 Core AMD Epic XXXX
Docker Version (`docker version`)	20.10.10
Docker-Compose Version (`docker-compose version`)	1.29.2
Reverse proxy (custom solution)	DEFAULT

Output of git diff origin/master, any other changes to the code? If so, please post them.


diff --git a/create_cold_standby.sh b/create_cold_standby.sh
index 924339af..600fe99c 100644
--- a/create_cold_standby.sh
+++ b/create_cold_standby.sh
@@ -2,6 +2,6 @@

export REMOTE_SSH_KEY=/root/.ssh/id_rsa
export REMOTE_SSH_PORT=22
-export REMOTE_SSH_HOST=my.remote.host
+export REMOTE_SSH_HOST=REPLACED

/opt/mailcow-dockerized/helper-scripts/_cold-standby.sh
diff --git a/data/conf/postfix/main.cf b/data/conf/postfix/main.cf
index a445b60c..ccf0062b 100644
--- a/data/conf/postfix/main.cf
+++ b/data/conf/postfix/main.cf
@@ -198,3 +198,6 @@ parent_domain_matches_subdomains = debug_peer_list,fast_flush_domains,mynetworks

# DO NOT EDIT ANYTHING BELOW #
# User overrides #

+myhostname = sub.example.tld
diff --git a/docker-compose.yml b/docker-compose.yml index d706d2dc..a0c7ca1e 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -80,7 +80,7 @@ services: environment:
- TZ=${TZ}
- IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
- IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
- IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
```
 - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
 - REDIS_SLAVEOF_PORT=${REDIS_SLAVEOF_PORT:-}
```
volumes: @@ -434,7 +434,7 @@ services: environment:
- TZ=${TZ}
- IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
- IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
- IPV6_NETWORK=${IPV6_NETWORK:-fd4d:6169:6c63:6f77::/64}
```
 - SNAT_TO_SOURCE=${SNAT_TO_SOURCE:-n}
 - SNAT6_TO_SOURCE=${SNAT6_TO_SOURCE:-n}
 - REDIS_SLAVEOF_IP=${REDIS_SLAVEOF_IP:-}
```
@@ -549,7 +549,7 @@ services:
- OLEFY_TMPDIR=/tmp
- OLEFY_PYTHON_PATH=/usr/bin/python3
- OLEFY_OLEVBA_PATH=/usr/bin/olevba3
- OLEFY_LOGLVL=20
- OLEFY_LOGLVL=10
  - OLEFY_MINLENGTH=500
  - OLEFY_DEL_TMP=1 networks: @@ -577,36 +577,6 @@ services: aliases:
    - ofelia
ipv6nat-mailcow:
depends_on:
- unbound-mailcow
- mysql-mailcow
- redis-mailcow
- clamd-mailcow
- rspamd-mailcow
- php-fpm-mailcow
- sogo-mailcow
- dovecot-mailcow
- postfix-mailcow
- memcached-mailcow
- nginx-mailcow
- acme-mailcow
- netfilter-mailcow
- watchdog-mailcow
- dockerapi-mailcow
- solr-mailcow
environment:
- TZ=${TZ}
image: robbertkl/ipv6nat
security_opt:
- label=disable
restart: always
privileged: true
network_mode: "host"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /lib/modules:/lib/modules:ro
networks: mailcow-network: driver: bridge
All third-party firewalls and custom iptables rules are unsupported. Please check the Docker docs about how to use Docker with your own ruleset. Nevertheless, iptabels output can help us to help you: iptables -L -vn, ip6tables -L -vn, iptables -L -vn -t nat and ip6tables -L -vn -t nat. ALL default
DNS problems? Please run docker exec -it $(docker ps -qf name=acme-mailcow) dig +short stackoverflow.com @172.22.1.254 (set the IP accordingly, if you changed the internal mailcow network) and post the output.
```
151.101.193.69
151.101.65.69
151.101.1.69
151.101.129.69
```

jmber commented 2 years ago

I can confirm this Issue. I think it is a problem between dovecot and sogo.

milkmaker commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

lastuptodate commented 1 year ago

I can also confirm this issue

mayk78 commented 1 year ago

I also experience this problem.

The cow has 16 GB and 6 cores , solr set to use 6 GB.
Mailboxes have been test imported with the cow syncjob , copy paste with trough thunderbird imap , PST outlook import in imap to try and exclude issues there. A second test cow also has this issue, and to exclude the hostingprovider/platform i tested locally in a kvm , same issue.

mailcowdockerized-sogo-mailcow-1 | Mar 15 17:43:10 26b5a76e4b8f sogod [168309]: [ERROR] <0x0x55628a0bb580[NGImap4Connection]> could not select URL: imap://user%40user.tld@172.22.1.250/INBOX/Tender/?tls=NO&tlsVerifyMode=none: {RawResponse = "{ResponseResult = {description = \"Mailbox doesn't exist: INBOXTender (0.002 + 0.001 + 0.001 secs).\"; result = no; tagId = 4; }; }"; reason = "Mailbox doesn't exist: INBOXTender (0.002 + 0.001 + 0.001 secs)."; result = 0; }

Running search again a few times may result in completing correct. Sometimes a search that runs correct being executed a second time fails with above error.
The mentioned Mailbox, that does not exist actually DOES exist. But it seems to quit randomly blurting out a random folder where it is searching at that moment so the mentioned mailbox is different each time.

root@vmi1110516:~# docker -v Docker version 23.0.1, build a5ee5b1 root@vmi1110516:~# docker-compose -v Docker Compose version v2.13.0 root@vmi1110516:~# Cow version 2023-03 (but having the issue also on previous versions.

nicolaspernoud commented 6 months ago

I also confirm this issue, it should not be marked as stale.

kcastner commented 6 months ago

Maybe we have to open it at sogo side too?

@nicolaspernoud: Would you like to open it at sogo, please? A reference to this issue should be fine.

nicolaspernoud commented 6 months ago

@kcastner : Hello, I did : https://bugs.sogo.nu/view.php?id=5912 . I wrote it in french since SoGO is now maintained by a french company. In the meantime, it would be nice to reopen the issue @andryyy or @FreddleSpl0it...

colpari commented 4 months ago

Also happening to us in version 2024-02. Did not happen in Version 2022-something we used before.

@FreddleSpl0it Search fails at random points. Does not seem related to resource conditions or timeouts (happens sometimes in the first 10 seconds of search, sometimes much later)

mailcow / mailcow-dockerized