search

modoboa / modoboa-imap-migration

An extension to ease the migration between 2 IMAP servers using offlineimap
ISC License
16 stars 12 forks source link

imap migration does not sync inboxes #10

Closed hadifarnoud closed 6 years ago

hadifarnoud commented 7 years ago

I followed instructions, emails are created after login but inbox sync does not happen. there is no error when I run the cron job manually via below command:

runuser -l vmail -c '/srv/modoboa/env/bin/python /srv/modoboa/instance/manage.py generate_offlineimap_config --output .offlineimaprc && /usr/local/bin/offlineimap'

.offlineimaprc

[general]
accounts = m.dalaei@farancell.com
pythonfile = ~/.offlineimap.py

[Account m.dalaei@farancell.com]
localrepository = Local_m.dalaei@farancell.com
remoterepository = Remote_m.dalaei@farancell.com
status_backend = sqlite

[Repository Local_m.dalaei@farancell.com]
type = IMAP
remotehost = localhost
remoteuser = m.dalaei@farancell.com
remotepasseval = get_user_password("m.dalaei@farancell.com")

[Repository Remote_m.dalaei@farancell.com]
type = IMAP
remotehost = mail.camva.ir
remoteport = 993
ssl = yes
remoteuser = m.dalaei@farancell.com
remotepasseval = get_user_password("m.dalaei@farancell.com")
readonly = true
hadifarnoud commented 7 years ago

I have done the following

root@email1:~# cp /srv/vmail/.offlineimaprc ~
root@email1:~# /usr/local/bin/offlineimap -o -d imap -d maildir
OfflineIMAP 6.6.1
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Debug mode: Forcing to singlethreaded.
Now debugging for maildir: Maildir repository debugging
Now debugging for : Other offlineimap related sync messages
kryskool commented 7 years ago

Hi

Can you tr to increase quota for each mailbox ?

Regards,

hadifarnoud commented 7 years ago

@kryskool the quota was 0MB by default. changed it to 1000MB and still sync doesn't work. not sure what's wrong as there is no error log.

Syslog:

Jan 31 06:58:30 email1 systemd[1]: Started User Manager for UID 0.
Jan 31 06:59:01 email1 CRON[22587]: (vmail) CMD (  $PYTHON $INSTANCE/manage.py handle_mailbox_operations)
Jan 31 06:59:09 email1 systemd[1]: Stopping User Manager for UID 0...
Jan 31 06:59:09 email1 systemd[22441]: Stopped target Default.
Jan 31 06:59:09 email1 systemd[22441]: Reached target Shutdown.
Jan 31 06:59:09 email1 systemd[22441]: Starting Exit the Session...
Jan 31 06:59:09 email1 systemd[22441]: Stopped target Basic System.
Jan 31 06:59:09 email1 systemd[22441]: Stopped target Sockets.
Jan 31 06:59:09 email1 systemd[22441]: Stopped target Paths.
Jan 31 06:59:09 email1 systemd[22441]: Stopped target Timers.
Jan 31 06:59:09 email1 systemd[22441]: Received SIGRTMIN+24 from PID 22601 (kill).
Jan 31 06:59:09 email1 systemd[1]: Stopped User Manager for UID 0.
Jan 31 06:59:09 email1 systemd[1]: Removed slice User Slice of root.
Jan 31 07:00:01 email1 CRON[22651]: (vmail) CMD (  $PYTHON $INSTANCE/manage.py handle_mailbox_operations)
Jan 31 07:00:01 email1 CRON[22650]: (root) CMD (   $PYTHON $INSTANCE/manage.py communicate_with_public_api)
Jan 31 07:00:01 email1 CRON[22655]: (vmail) CMD (  cd /srv/vmail && $PYTHON $INSTANCE/manage.py generate_offlineimap_config --output .offlineimaprc && /usr/local/bin/offlineimap > /dev/null 2>&1)
Jan 31 07:00:01 email1 CRON[22656]: (root) CMD (   $PYTHON $INSTANCE/manage.py logparser &> /dev/null)
Jan 31 07:00:01 email1 CRON[22653]: (root) CMD (   $PYTHON $INSTANCE/manage.py generate_rights)
Jan 31 07:00:01 email1 CRON[22660]: (root) CMD (   $PYTHON $INSTANCE/manage.py modo check_mx)
Jan 31 07:01:01 email1 CRON[22751]: (vmail) CMD (  $PYTHON $INSTANCE/manage.py handle_mailbox_operations)
Jan 31 07:01:14 email1 systemd[1]: Created slice User Slice of root.
Jan 31 07:01:14 email1 systemd[1]: Starting User Manager for UID 0...
Jan 31 07:01:14 email1 systemd[1]: Started Session 13652 of user root.
Jan 31 07:01:14 email1 systemd[22773]: Reached target Paths.
Jan 31 07:01:14 email1 systemd[22773]: Reached target Sockets.
Jan 31 07:01:14 email1 systemd[22773]: Reached target Timers.
Jan 31 07:01:14 email1 systemd[22773]: Reached target Basic System.
Jan 31 07:01:14 email1 systemd[22773]: Reached target Default.
Jan 31 07:01:14 email1 systemd[22773]: Startup finished in 18ms.
Jan 31 07:01:14 email1 systemd[1]: Started User Manager for UID 0.
Jan 31 07:02:01 email1 CRON[22872]: (vmail) CMD (  $PYTHON $INSTANCE/manage.py handle_mailbox_operations)
Jan 31 07:02:01 email1 CRON[22873]: (root) CMD (   $PYTHON $INSTANCE/manage.py generate_rights)
Jan 31 07:03:01 email1 CRON[22926]: (vmail) CMD (  $PYTHON $INSTANCE/manage.py handle_mailbox_operations)
Jan 31 07:04:01 email1 CRON[22953]: (root) CMD (   $PYTHON $INSTANCE/manage.py generate_rights)
Jan 31 07:04:01 email1 CRON[22955]: (vmail) CMD (  $PYTHON $INSTANCE/manage.py handle_mailbox_operations)
Jan 31 07:05:01 email1 CRON[23014]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Jan 31 07:05:01 email1 CRON[23016]: (root) CMD (   $PYTHON $INSTANCE/manage.py logparser &> /dev/null)
Jan 31 07:05:01 email1 CRON[23017]: (vmail) CMD (  $PYTHON $INSTANCE/manage.py handle_mailbox_operations)
root@email1:~#
hadifarnoud commented 7 years ago

I think there is an issue with SSL certificate?

root@email1:~# /usr/local/bin/offlineimap --info -o -d imap -d maildir

OfflineIMAP 6.6.1
  Licensed under the GNU GPL v2 or any later version (with an OpenSSL exception)
Debug mode: Forcing to singlethreaded.
Now debugging for maildir: Maildir repository debugging
Now debugging for : Other offlineimap related sync messages
Traceback (most recent call last):
  File "/usr/local/bin/offlineimap", line 23, in <module>
    oi.run()
  File "/usr/share/offlineimap/offlineimap/init.py", line 52, in run
    self.__serverdiagnostics(options)
  File "/usr/share/offlineimap/offlineimap/init.py", line 429, in __serverdiagnostics
    account.serverdiagnostics()
  File "/usr/share/offlineimap/offlineimap/accounts.py", line 197, in serverdiagnostics
    self.ui.serverdiagnostics(remote_repo, 'Remote')
  File "/usr/share/offlineimap/offlineimap/ui/UIBase.py", line 461, in serverdiagnostics
    folders = repository.getfolders()
  File "/usr/share/offlineimap/offlineimap/repository/IMAP.py", line 384, in getfolders
    imapobj = self.imapserver.acquireconnection()
  File "/usr/share/offlineimap/offlineimap/imapserver.py", line 483, in acquireconnection
    tls_level=self.tlslevel,
  File "/usr/share/offlineimap/offlineimap/imaplibutil.py", line 186, in __init__
    super(WrappedIMAP4_SSL, self).__init__(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 2138, in __init__
    IMAP4.__init__(self, host, port, debug, debug_file, identifier, timeout, debug_buf_lvl)
  File "/usr/lib/python2.7/dist-packages/imaplib2.py", line 357, in __init__
    self.open(host, port)
  File "/usr/share/offlineimap/offlineimap/imaplibutil.py", line 193, in open
    "having SSL helps nothing.", OfflineImapError.ERROR.REPO)
offlineimap.error.OfflineImapError: No CA certificates and no server fingerprints configured.  You must configure at least something, otherwise having SSL helps nothing.
tonioo commented 7 years ago

@hadifarnoud I realize that the git tree contains modifications not packaged yet, on especially one related to this SSL issue. Have you tried with git version?

hadifarnoud commented 7 years ago

I did update it to latest version as you said @tonioo. still same issue ☹️

hadifarnoud commented 7 years ago

ideas guys?

tonioo commented 7 years ago

@hadifarnoud Maybe you can ask on offlineimap mailing list? Otherwise, you also take a look at professional services for modoboa.

hadifarnoud commented 7 years ago

thanks @tonioo. will pay for a support ticket to fix this

Nodeswitch commented 7 years ago

Throughout a bunch of trial and error today, using v6.6.1 and v7.1.1 of offlineimap, we've been able to get mail to sync by commenting out the "pythonfile" and "status_backend" variables in .offlineimaprc.

Issue #16 has been raised for "status_backend" no longer being supported from v7.1.0 and beyond, but we're still having issues using v6.6.1.

With pythonfile and status_backend uncommented, a connection is made to the remote server, but that's it - nothing else happens. No errors are produced.

Edit: Forgot to add that we had to comment out "remotepasseval" when commenting out the other two.

dugite-code commented 7 years ago

I am seeing the same issue as @Nodeswitch with the .offlineimap.py file. After changing remotepasseval to remotepass and writing .offlineimap.py line by line offlineimap begins to fail once django.setup() is called. Running manually doesn't cause any issues.

Knowing this I looked into what django.setup() actually does. From my understanding the helper script only needs to load apps and settings

This seems to work, however I have never really worked with django and might be missing something

import os
import site
import sys

site.addsitedir("/srv/modoboa/env/lib/python2.7/site-packages")
sys.path.append("/srv/modoboa/instance")
os.environ["DJANGO_SETTINGS_MODULE"] = "instance.settings"

import django
from django.apps import apps
from django.conf import settings
apps.populate(settings.INSTALLED_APPS)

from modoboa_imap_migration.models import Migration

def get_user_password(username):
    """Retrieve a password from Modoboa's database."""
    return Migration.objects.select_related().get(
        mailbox__user__username=username
    ).password

Note: Debian 9, offlineimap v7.0.12, imaplib2 v2.55 (system), Python v2.7.13