feat: Gmail / IMAP importer (commandline only)

[nameduser0]

Uses offlineimap to allow you to:

• Import specific remote Gmail/IMAP folders to your MIAB mailbox (yes it preserves your Gmail labels)
• Export your MIAB mailbox folders to Gmail/IMAP (yes it also preserves your Gmail labels)
• View local & remote folders/labels

So yes you can use it to migrate from one Gmail account to another via MIAB! Currently draft because the export hasn't been tested yet, but import works.

Has been crafted so that it could potentially be extended with a UI.

Process:

1. User enters login details and import/export
2. Info mode to get the folder lists and check authentication
3. User selects folders
4. Run with progress percentage output

Folder mappings are fixed and reciprocal Plenty of guides out there on how to get a Gmail POP/IMAP password (no it's not the one you use to log in with normally)

In response to this: https://support.google.com/a/answer/2855120

[nameduser0]

Export currently at 36.2% on my mailbox..

[Programming4People]

The export may be stuck due to throttling by google...

explained here: https://support.google.com/a/answer/1071518

just wasted a day shuttling between google support for admin, migration and email team (three seperate teams) and they say that the problem is due to throttling and maxing out day downloads....

[ddavness]

I wonder how useful would it be to support something like https://takeout.google.com - basically you can use it to ask Google to crunch your entire mailbox into a .zip file. Though it will take a while until you can download that file (depends on how full your mailbox is), but after that there will be no throttling since we'll be just using that file.

[Programming4People]

Absolutely interesting.. I have been shuttling now for 3 days attempting to get a format of what the data looks like.. Google has great tools for IMPORTING.. but they are like hotel California.. you can check in ... but you will jump through hoops before you can leave....

I shall have data for one domain... so I can see...

Another issue is you can only do one export every 30 days.. so you have to have mailserver all set up.. It is easy to download a list of google apps users... but aliases? groups? no way.. you have to do by hand.. insane.

Another issue is that Legacy had a 15G max size storage... I have spent the entire day dealing now with Migration group.. and they are seeing if I try to do an export of a domain that has a TOTAL greater then 15 G will it work. We have one domain that has several accounts that are close to the limit...

(this is the question that google's migration has been working on solving)

---

QUOTE so.. just to ensure I am clear.. if we have 4 users.. close to 14 gigs each in space.. and admin does export... (so total is close to 56G) are we able to download entire space... as admin that is also limited to 15G? END QUOTE

---

the above is what I am dealing with with Google..... supposedly humans.. but moved from group to group in circle..

---

UPDATE: Google Workspace Support, Aditya 7:48 PM wrote: "Yes, you will be able to."

so looks good on space...

[Programming4People]

Again just got confirmation from same person ( Adita) that depending on mail messages

"The emails will be in .pst or .mbox" 7:53 PM ME-> ok. thanks.. and what decides on the difference?

Google Workspace Support, Aditya 7:54 PM It depends on the size of the file

Google Workspace Support, Aditya7:57 PM Apart from this, is there anything else I can help you with? 7:57 PM ME -> and just to confirm.. you are with what team? Google Workspace Support, Aditya7:57 PM I am from the Admin team.

[nameduser0]

I think you've missed the point of this addon. It uses IMAP to download (import) each gmail message in maildir format to a MIAB mailbox. If it fails for whatever reason it will resume. It also downloads the message labels.

Then if you want you can upload/export to another gmail mailbox.

My export wasn't stuck, just reporting that export was working.

[Programming4People]

I think you've missed the point of this addon. It uses IMAP to download (import) each gmail message in maildir format to a MIAB mailbox. If it fails for whatever reason it will resume. It also downloads the message labels.

Then if you want you can upload/export to another gmail mailbox.

My export wasn't stuck, just reporting that export was working.

I did not see an update to say your transfer was done... hence my ping re google throttling and transfer limitations. I used and imap copy and had one account locked down, as download was too big and got nailed by google for hitting max download/day David commented that there may be an easier way of pulling info via google data export. I was worried and have been trying to get an answer from google apps team on potential limitations, as they only allow 1 export of accounts every 30 days... and had fears that there would be problems with 15 gig total limit of cloud drive.. but.. supposedly it is not an issue at the moment.. Shall know more tomorrow as first big transfer happens.

[nameduser0]

The import was already complete. Export also completed. IMAP should also resume once your daily limit is reset.

[Programming4People]

so after starting export.. next surprise: from 72 hrs to 9 days....

email from google:

Data export for DOMAIN successfully started

Hello,

A super administrator of your Google Account has started exporting data for DOMAIN. Data export typically takes 72 hours but can take up to 9 days, depending on the size of your data export. You'll get a confirmation email when the export is complete. You can see the current status of the export in the Data Export tool.

For more information about the Data export tool, see Export your organization’s data.

Sincerely, The Google Workspace Team

---

[casesolved-co-uk]

@ddavness I've rebased onto your latest changes. Are there any issues with this?

[ddavness]

Taking a look! :)