mylesagray / blog-comments

Comments for Blah, Cloud. Hugo blog
0 stars 0 forks source link

Migrating from Google Workspaces to iCloud+ | Blah, Cloud #55

Open utterances-bot opened 2 years ago

utterances-bot commented 2 years ago

Migrating from Google Workspaces to iCloud+ | Blah, Cloud

Google recently accounced that it was getting rid of the legacy free tier for Google Apps/Workspace accounts, something I've used for 10+ years, and I don't care for that. I already pay for iCloud+ and it allows custom domains, so let the migration begin!

https://blah.cloud/miscellaneous/migrating-google-workspaces-to-icloud-custom-domain/

mstrchng commented 2 years ago

Thanks for putting this together! Bit confused about setting up email forwarding on "my target Gmail account to the iCloud acccount’s address" -- aren't they the same email address?

Is there a way to check/view the migrated messages on iCloud while it's still in progress?

Thanks again - really looking forward to getting through this.

mylesagray commented 2 years ago

Howdy!

So no they should be different, your gmail address is something like you@yourdomain.com and the iCloud domain is you@icloud.com.

iCloud will only take ownership of @yourdomain.com after the whole process is complete (it is essentially aliasing your domain and addresses to the @icloud.com mailbox).

And yup, while imapsync is running you can check the migrates mail in iCloud Mail.

benw-git commented 2 years ago

WOW! THANK YOU for this extremely helpful, detailed post. And timely, I started looking for such info a few days ago, and you just posted!

I'm in an identical situation as you are, but with much less experience and knowledge in this space. I'm plenty tech savvy, so totally understand all you've described, but never would have found these tools and figured out for myself the way you have, and you've smoothed out a lot of potential speed bumps for me, too. Looking forward to putting it to good use soon.

I don't mean to clog your comments with praise, but you've so specifically and thoroughly addressed my issue, I have to let you know.

Docker etc is new to me, so a bit of research/learning to be done for me there. And I'll be running on my wife's MacBook Air since no NAS. Have only used SSH a tiny bit so will see what happens with session time-outs you mention, hoping goes smoothly. Would consider setting up a virtual server (like from vultr) but I assume there would be a fair amount of config/install etc that I wouldn't be prepared for those details, so guess my Mac will have to do.

If you have any further suggestions/comments for my situation I'm all ears and appreciation.

Thx again --Ben

mylesagray commented 2 years ago

@benw-git thats fantastic, I'm really pleased it's going to help someone else out!

Docker is an incredible tool, and like you say, it'll run just fine on your wife's MacBook, just make sure it's plugged in to power (and Ethernet if you can) and turn sleep off, and let it rip.

Another option (I've never heard of the platform you mentioned) is to use Linode and get a cheap little VPS for a month, and just run it from there, it'll teach you a bunch about Linux, SSH, Docker, etc.

However, there's a lot to be said for not stressing about the "how" and instead just getting it done ;)

blackcat1975 commented 2 years ago

This is so great and exactly what I need to do but I feel slightly out of my depth. I really want to keep everything in the cloud so this is a great option. Just trying to get my head around docker and the imapsync. So I would load docker either on my Mac or my spare Ubuntu machine. Use the docker pull command to have the application running in docker. Test it with the justlogin command and then use your settings/script? Will this bring in all folders and file structure I have in the current mailbox?

mylesagray commented 2 years ago

@blackcat1975 just to be a little more explicit, yes you would install Docker on a machine you have - there is no need to do a docker pull as the docker run command does that implicitly.

But yeah, the general idea would be to run with the justlogin flag to ensure that it all auths correctly.

As to your question around folders - I actually don't use folders at all, I rely on archiving and search for everything, however, when running justlogin you should see the folder mapping as part of the output, make sure that matches what you're after.

Maybe give the imapsync docs a little look, if you want some more examples: https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt (check out the bit in "Categories" that might be what you're after)

colinpearse commented 2 years ago

Just a follow on from mstrchng's question. I use the same email from both my Gsuite and Apple logins. When I pay for iCloud will I automatically get an myname@icloud.com email, or do I have to set up a different email for Apple temporarily so I can copy the emails?

mylesagray commented 2 years ago

Hey @colinpearse - you should automatically get an @icloud.com address yes, that is what I'm referring to here. For clarity, my start and end addresses were both the same, I just used the iCloud.com address for the forwarding rule temporarily.

grahamr commented 2 years ago

Any chance you could share a redacted screenshot of what the mail app looks like once done? Do you access your custome domain mail in its own account, or a sub folder? How is it identified? (As someone with a gmail, workspace and iCloud all on the go)

mylesagray commented 2 years ago

Hey @grahamr,

The custom domain(s) are just part of your "iCloud" account, there is no distinction between them, see below:

image

You can then choose, in Mail settings what your default "from" address is, so I changed mine to one of my migrated addresses.

HTH

dfbills commented 2 years ago

This was a fantastic tutorial. Exactly what the doctor ordered. Thank you so much! Worked like a charm for me.

gramoscarr commented 2 years ago

Great write up, any chance you can confirm if group email addresses can be used in iCloud. I was thinking that maybe the alias feature might allow 2 different accounts to use the same alias essentially creating a group email.

mylesagray commented 2 years ago

@gramoscarr unfortunately it seems to be a 1-1 mapping, no shared mailboxes.

gramoscarr commented 2 years ago

that's unfortunate, thanks for the quick response. Maybe one day soon I hope.

davidgustav commented 2 years ago

This was really helpful. Thank you so much. I wish I found it before dragging some emails from Inbox to Inbox in Apple Mail and gotten a perfect sync instead of a minor mismatch.

Re: the deltas, I found that I could still run the sync even after DNS cutover b/c the servers are not specific to your domain. It's using a gmail IMAP server. So as long as you don't close the Workspace account right away, you should still be able to run the process again for a final sync without worrying about forwarding in the short term. I did the bulk of the sync before DNS cutover, but once I verified I was getting new emails to iCloud, I ran it again to pull in the final stragglers.

rodor1155 commented 2 years ago

Superb write up!

I unfortunately discovered this AFTER I completed the DNS migration. Do you know of any options that will help me migrate from a Gmail account to my new iCloud account?

mylesagray commented 2 years ago

@rodor1155 unfortunately not, because you'd have to own gmail.com - which you don't - but I'm in the same position as you.

That said, for me, it's my trash/throwaway account - so I may just leave it and migrate stuff I care about over time.

rockdirector commented 2 years ago

Thanks for this helpful guide.

For those who don’t really use or need Docker, imapsync is also available in homebrew: https://formulae.brew.sh/formula/imapsync

dfbills commented 2 years ago

Many thanks, @rockdirector! Using it in docker was fun until I saw how much system resource it was taking in the background while doing "nothing."

onemico commented 2 years ago

Great guide, thanks. I could probably work this out from your guide, but my wife uses my custom domain for her email too, setup in Gmail just like mine (I just created her an account). Can I just follow this guide twice for the 2 accounts? Will I need to set her up a sub-account in iCloud for her email address on my iCloud instead of using her own?

mylesagray commented 2 years ago

@onemico good Q, I believe how this works is you need to be in an Apple "Family": https://support.apple.com/en-gb/guide/icloud/mme8ed800b5d/icloud and then you can create the email under your wife's account.

It appears that following this twice would be adequate.

onemico commented 2 years ago

@mylesagray thanks. Looks interesting. Guess I've got some easter weekend plans now :)

JetLagJunkie commented 2 years ago

Currently I have multiple family members on our custom domain account at Google. I have started migrating my account, but I need to migrate everyone else's account as well. Do I need to add an iCloud email to everyones Apple account prior to setting up the custom domain email for each of them. I assume I do. What did everyone else have to do?

mylesagray commented 2 years ago

You don’t have to set up the custom domain on their accounts first, you can do the migration and then do a final sync post domain addition.

On 16 Apr 2022, at 17:14, JetLagJunkie @.***> wrote:

 Currently I have multiple family members on our custom domain account at Google. I have started migrating my account, but I need to migrate everyone else's account as well. Do I need to add an iCloud email to everyones Apple account prior to setting up the custom domain email for each of them. I assume I do. What did everyone else have to do?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

dfbills commented 2 years ago

While we’re all discussing this and very much of a like mind about the situation, has anyone seen any details on what will be included in the “free version” of Google Workspaces beyond?

  1. The barebones signup form
  2. The fact that there will be “something”
  3. Will not include premium features like custom email or multi-account management

My best guess is that “Google Apps for Your Domain” will basically be downgraded to Gmail-only- although basic consumer Gmail accounts are still eligible for things like Docs, Drive, and even Google Voice.

colinpearse commented 2 years ago

Has anyone encountered the following limitation? We are a family of 4, so have 4 emails in our domain, but Apple has a 3 emails per domain limit. This seems crazily stingy (when google offers 100 email to forward on their domain service), does anyone know a way around this?

davidgustav commented 2 years ago

@colinpearse I think that's a 3 email per user limit. You would add each family member to your family in Apple and then each can have an email address at the domain. When you set up the custom domain you're prompted to select if everyone in the family can have an email address at the domain or just you. You would select everyone.

colinpearse commented 2 years ago

@davidgustav thanks for your response. I think I didn't understand the "one apple ID as domain manager" -> "configure separate apple IDs for each family member" -> "then I can have up to 3 emails per family member". I thought I could manage multiple childrens' emails under one Apple ID, but when I need a couple of emails that 3 limit is quickly breached. Thanks for your help.

Gnindit commented 2 years ago

JetLagJunkie When I wanted to add my family members to my apple family sharing the phones of each one asked to add an icloud email first. So I just did that in the phone. I think the email needs to be there but will not be used later anyway.

jayleswo commented 2 years ago

@mylesgray any concerns about security? Allowing imapsync access to years of emails to sync them with iCloud has me a little concerned....

mylesagray commented 2 years ago

Yeah, that’s understandable, if you are worried you can always run an app like LittleSnitch or equivalent to make sure imapsync can only talk to and from iCloud and Gmails mail servers.

It’s a pretty long standing OSS project, so I was happy to trust the author.

Sent from mobile

On 20 Apr 2022, at 20:13, jayleswo @.***> wrote:

 @MylesGray any concerns about security? Allowing imapsync access to years of emails to sync them with iCloud has me a little concerned....

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

julian-tr commented 2 years ago

This is very helpful, thank you.

Is it correct to say that iCloud doesn't treat each of the domain email addresses as separate accounts/mailboxes, and everything (your original name@icloud.com email plus the new name@yourdomain.com email address) is lumped together in the original single iCloud mailbox?

ehre commented 2 years ago

Thanks for a great post! Out of curiosity, since you first sync to name@icloud.com, is there any point in going through the iCloud custom domain setup up to step 2 before running imapsync as you suggest, or would the end result be the same if you first sync to the icloud address and then setup the domain at a later stage?

mylesagray commented 2 years ago

Good point, AFAIK the order is unimportant - you should be able to set up the custom domain after running imapsync

On 28 Apr 2022, at 12:26, Hans Ehrencrona @.***> wrote:

Thanks for a great post! Out of curiousity, since you first sync to @.***, is there any point in going through the iCloud custom domain setup up to step 2 before running imapsync as you suggest, or would the end result be the same if you first sync to the icloud address and then setup the domain at a later stage?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

ehre commented 2 years ago

Thanks again, Myles. I have now transferred 10+ GB of emails, and I can confirm that everything worked fine when setting up the custom domain after all emails were transferred.

Also thanks for your tip on DMARC! I might add that I now get perfect scores for SPF/DKIM/DMARC when sending a test email from the Mac or iOS Mail apps, but if I do the same thing from the iCloud web interface, I get a deduction on "no DKIM signature included" which is a bit surprising. My favorite email client Spark unfortunately handles the custom domain as an alias, which leads to a DMARC fail.

wzaremba commented 2 years ago

Quick question, what NAS do yo use and how did you run this sync process there? I have Synology and running containers with lot's of parameters is a pain.

mylesagray commented 2 years ago

I run a Synology too, but I just SSH into it and run the Docker commands - it shows right up in the UI even if you do that.

Sent from mobile

On 12 May 2022, at 15:18, Wojciech Zaremba @.***> wrote:

 Quick question, what NAS do yo use and how did you run this sync process there? I have Synology and running containers with lot's of parameters is a pain.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

wzaremba commented 2 years ago

And this executes when you disconnect? I need to try that. I have get rid of most of my legacy mails but my wife says she is tied to emails and she does not want to delete them. So we are moving :)


Od: Myles Gray @.> Wysłane: Thursday, May 12, 2022 7:57:16 PM Do: mylesagray/blog-comments @.> DW: Wojciech Zaremba @.>; Comment @.> Temat: Re: [mylesagray/blog-comments] Migrating from Google Workspaces to iCloud+ | Blah, Cloud (Issue #55)

I run a Synology too, but I just SSH into it and run the Docker commands - it shows right up in the UI even if you do that.

Sent from mobile

On 12 May 2022, at 15:18, Wojciech Zaremba @.***> wrote:

 Quick question, what NAS do yo use and how did you run this sync process there? I have Synology and running containers with lot's of parameters is a pain.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

— Reply to this email directly, view it on GitHubhttps://github.com/mylesagray/blog-comments/issues/55#issuecomment-1125274239, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKTG66QYGRUTHQ7AJKBAW33VJVA7ZANCNFSM5M4IKBAA. You are receiving this because you commented.Message ID: @.***>

mylesagray commented 2 years ago

If you run it in daemon mode, with -d yup!

Sent from mobile

On 12 May 2022, at 19:20, Wojciech Zaremba @.***> wrote:

 And this executes when you disconnect? I need to try that. I have get rid of most of my legacy mails but my wife says she is tied to emails and she does not want to delete them. So we are moving :)


Od: Myles Gray @.> Wysłane: Thursday, May 12, 2022 7:57:16 PM Do: mylesagray/blog-comments @.> DW: Wojciech Zaremba @.>; Comment @.> Temat: Re: [mylesagray/blog-comments] Migrating from Google Workspaces to iCloud+ | Blah, Cloud (Issue #55)

I run a Synology too, but I just SSH into it and run the Docker commands - it shows right up in the UI even if you do that.

Sent from mobile

On 12 May 2022, at 15:18, Wojciech Zaremba @.***> wrote:

 Quick question, what NAS do yo use and how did you run this sync process there? I have Synology and running containers with lot's of parameters is a pain.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

— Reply to this email directly, view it on GitHubhttps://github.com/mylesagray/blog-comments/issues/55#issuecomment-1125274239, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKTG66QYGRUTHQ7AJKBAW33VJVA7ZANCNFSM5M4IKBAA. You are receiving this because you commented.Message ID: @.***> — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

JetLagJunkie commented 2 years ago

So I have run imapsync multiple times with what appears to be success, but after I re-sync the same accounts again, some of the recent email is not synced even though the process completed to success. I noticed that when it runs the process, that it shows what emails are being synced, but it has a future date, what does that mean?

mylesagray commented 2 years ago

That’s a good one – i’ve never noticed that, perhaps the server you’re syncing from/to are behind you TZ wise? Or the src/dst servers aren’t using the same TZ? Just guessing.

JetLagJunkie commented 2 years ago

Yeah, I thought about that, but in some cases it is days into the future, so it is not a TZ thing.

wrast commented 2 years ago

Super helpful in migrating a couple of gmail domains I have over to iCloud. There were some emails that didn't migrate due to being "too big". Also, it appears that emails with multiple gmail labels on them only go into a single folder, most of the time this happened to be "Inbox". I wouldn't mind have multiple copies of the email if it would maintain the label->folder mapping during the sync.

tromano206 commented 2 years ago

ok, so I saw this AFTER I changed my dns etc and now I am worried. I am fine with leaving the messages in the cloud/ I have a bunch of messages in my third party email apps (spark) or mail.app so I don't necc want to migrate but I don't want them to disappear or to lose access. It doesn't seem to have updated yet so I'm not sure what to do/what I did wrong.

sbkp commented 2 years ago

Thanks so much for this! Just starting the process. Do gmail messages flagged as "important" get synced twice to Apple? (One in the important folder and once as part of all mail?)

mylesagray commented 2 years ago

@sbkp - From what I remember no, just the once - "important" isn't a folder, just a label

sbkp commented 2 years ago

Thanks, Myles. I'm new to iCloud Mail and now sure how labels migrate. imapsync created a folder in my iCloud mail account, so I got concerned.

samAnnisBrown commented 1 year ago

Thanks Myles - as a tech guy with 2 young kids, I've wanted to migrate since I saw the Google announcement I'd have to move to a paid account, but have struggled to find the time to work it out. <10 mins configuring app passwords and launching a container, and I'm happily migrating over on my UnRAID box. Thanks for the succinct walk-through.

grahamr commented 1 year ago

@samAnnisBrown - When you used unraid did you use CA to add the docker from Docker Hub? How did you set up all the variables - im temped to run on a local docker but would just rip this through my unraid box if it could nail it down easier

andrew-curtis commented 1 year ago

Thank you, for this!

For anyone else that was having issues with getting the docket container going on their Synology too, I thought I would leave the following notes.

I don’t know what I was doing wrong but I couldn’t get it going from the command line when using the example docker run command.

I use Portainer for managing my docker instances anyway so I just needed to tweak the example docker command slightly to work with Portainer’s CMD parameter when adding a new container so I could add a new imapsync instance successfully.

Replace docker run gilleslamiral/imapsync imapsync with /usr/bin/imapsync in the CMD parameter and you should be golden.

E.g. /use/bin/imapsync --errorsmax 200 --user1 {gmail-email-address} --password1 {app-specific-password-for-gmail} --host2 imap.mail.me.com --ssl2 --user2 {icloud-email-address} --password2 {app-specific-password-for-icloud} --gmail1

I hope someone else finds this useful too. 😊