search

OfflineIMAP / offlineimap

Read/sync your IMAP mailboxes (python2) [LEGACY: move to offlineimap3]
http://www.offlineimap.org
Other
1.78k stars 361 forks source link

Offlineimap not syncing all folders/messages #687

Open benjaminpreiss opened 3 years ago

benjaminpreiss commented 3 years ago

General informations

Configuration file offlineimaprc

nr1: configuration server1 -> local maildir nr2: configuration local maildir -> server2

pythonfile (if any)

NO pythonfile

Step 1: Clean Slate

rm -r ~/.offlineimap/Account-Account1 ~/.offlineimap/Repository-RemoteAccount1 ~/.offlineimap/Repository-LocalAccount1 ~/.offlineimap/Account-Account2 ~/.offlineimap/Repository-RemoteAccount2 ~/.offlineimap/Repository-LocalAccount2 ~/.offlineimap/Account-Account3 ~/.offlineimap/Repository-RemoteAccount3 ~/.offlineimap/Repository-LocalAccount3
rm -r ~/linux-files-wsl/Account1 ~/linux-files-wsl/Account2 ~/linux-files-wsl/Account3

Step 2: Info Server1 -> local maildir

offlineimap -a Account1,Account2,Account3 -c offlineimap_from_server1.conf --info

Logs

Step 3: Dry Run Server1 -> local maildir

offlineimap -a Account1,Account2,Account3 -c offlineimap_from_server1.conf --dry-run

Logs

Step 4: Real Run Server1 -> local maildir

offlineimap -a Account1,Account2,Account3 -c offlineimap_from_server1.conf

Logs

Step 5: Info local Maildir -> Server2

offlineimap -a Account1,Account2,Account3 -c offlineimap_to_server2.conf --info

Logs

Step 6: Dry Run local Maildir -> Server2

offlineimap -a Account1,Account2,Account3 -c offlineimap_to_server2.conf --dry-run

Logs

Step 7: Clean offline imap cache (because of UID validity problems)

rm -r ~/.offlineimap/Account-Account1 ~/.offlineimap/Repository-RemoteAccount1 ~/.offlineimap/Repository-LocalAccount1 ~/.offlineimap/Account-Account2 ~/.offlineimap/Repository-RemoteAccount2 ~/.offlineimap/Repository-LocalAccount2 ~/.offlineimap/Account-Account3 ~/.offlineimap/Repository-RemoteAccount3 ~/.offlineimap/Repository-LocalAccount3

Step 8: Real Run local Maildir -> Server2

offlineimap -a Account1,Account2,Account3 -c offlineimap_to_server2.conf

Logs

Problemdescription

Sadly, some messages are simply not transferred to server 2. E.g. RemoteAccount3:INBOX UID 3 and 4 are missing.

The same happens for all messages in the following folders:

I suspect this has something to do with my nametrans rules... But I am not quite sure because it works mostly.

ssclift commented 3 years ago

I hit a similar problem transferring mail to a new server, but I think it was old data in the meta-files. I've not traced this completely, I suspect it has to do with with the way meta-data is stored by account name. I cleaned out the meta-data and it worked. That's not really a solution for everyday use, however.

benjaminpreiss commented 3 years ago

@ssclift Thanks mate, how do you clean out the meta-data?

ssclift commented 3 years ago

There's a *.meta directory. However, I don't really understand what I did there, it just worked...

I suspect if your configuration for server1->local and the configuration for local->server2 use completely different account names in the [Accounts] section of the offlineimap.conf, it might do the trick.

Removing the meta-data is a messy solution... I was in a hurry. Again, it was just a guess without really understanding the guts.

ssclift commented 3 years ago

Ah ha... I've just had to set up similarly to you after switching email servers.

I used separate meta-data directories, and that seems to work.

[general] accounts = ... metadata = ~/mailxfer/fetch_old.meta

then another config file with

[general] accounts = ... metadata = ~/mailxfer/update_new.meta

Works for me, your mileage may vary.

benjaminpreiss commented 3 years ago

Wow, thank you so much! Will try and get back to you!

benjaminpreiss commented 3 years ago

@ssclift Sadly, it didn't work :( Same folders and mails missing, exactly the same output as before...

I wonder whether directly syncing from imap to imap would do the trick?

Also, I read in the docs that nametrans mapping has to be unique, that means merging folders should not be possible?

nicolas33 commented 3 years ago

Yes, merging folders is not possible. There's a one-to-one mapping between local and remote with the UIDs for the emails for each mailbox.

nicolas33 commented 3 years ago

What you're attempting to do should be possible. HOWEVER, be sure to not mix the cache data between both syncs setups. Either use different conf file files with uniq metadatadir OR have one config file and two different accounts having uniq remote and local repository sections each.