timmcmic / DLConversionV2

MIT License
44 stars 9 forks source link

If Thread1 fails during a batch migration, all other threads stall for a long time #53

Closed MassimoPascucci closed 2 years ago

MassimoPascucci commented 2 years ago

During a batch migration, at most 5 threads are running concurrent migration; however, only the first one is responsible for triggering Active Directory replication and ADConnect synchronization. This is documented here: https://timmcmic.wordpress.com/2021/09/02/office-365-distribution-list-migrations-version-2-0-part-9.

If Thread1 fails end exits because of any error, all the other threads stall waiting for changes to appear in Azure AD, but no one is actually triggering an ADConnect sync. This should eventually resolve on its own, but depending on AD replication settings and ADConnect sync settings it might take quite a while.

timmcmic commented 2 years ago

Massimo:

This is acknowledged as you have pointed out. Thread one is the protected thread. I tried working in additional ad replication and sync - but you'd be surprised with the number of customers that all ready question the replication to begin with.

The guidance suggests that you should always be specifying the GC that the ad connect connector points to for reading objects. If thread 1 fails that at least lowers the ad replication issue.

For ad connect - if thread 1 fails you're looking at anywhere from 0 minutes to 30 minutes assuming that you're using a standard ad connect replication cycle. So all DL migrations could be hung up longer than expected.

It's a trade off between reliability and time (and the likelihood of the first thread failing).

Powershell does not lend itself to multi-threading with coordination very easily - when you look at the code what I have here I felt was fairly complex to begin with - so considering the other factors I did not work to make it more complex.

Tim

============================== Timothy J. McMichael Senior Support Escalation Engineer @.**@.> (980)-776-7465

Hours: Sunday - Wednesday 08:00 - 16:00 eastern time zone.

Manager: Tom Roughley @.**@.>)

Premier Support - (800)-936-3100 Broad Commercial Support - (800)-936-4900

==============================

From: Massimo Pascucci @.> Sent: Tuesday, May 10, 2022 12:31 PM To: timmcmic/DLConversionV2 @.> Cc: Subscribed @.***> Subject: [timmcmic/DLConversionV2] If Thread1 fails during a batch migration, all other threads stall for a long time (Issue #53)

During a batch migration, at most 5 threads are running concurrent migration; however, only the first one is responsible for triggering Active Directory replication and ADConnect synchronization. This is documented here: https://timmcmic.wordpress.com/2021/09/02/office-365-distribution-list-migrations-version-2-0-part-9https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftimmcmic.wordpress.com%2F2021%2F09%2F02%2Foffice-365-distribution-list-migrations-version-2-0-part-9&data=05%7C01%7Ctimmcmic%40microsoft.com%7C0e66232a70d54da2f57708da32a26aa7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637877970406561835%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=w50rYwdnIuTe7kI7PUss12ymNnkkag%2B4XKRNZM3qdfM%3D&reserved=0.

If Thread1 fails end exits because of any error, all the other threads stall waiting for changes to appear in Azure AD, but no one is actually triggering an ADConnect sync. This should eventually resolve on its own, but depending on AD replication settings and ADConnect sync settings it might take quite a while.

- Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Ftimmcmic%2FDLConversionV2%2Fissues%2F53&data=05%7C01%7Ctimmcmic%40microsoft.com%7C0e66232a70d54da2f57708da32a26aa7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637877970406561835%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=AcVTfWyeEzJ3Gy%2BaDSJIp6LdEmuK23UAhJWvccSekoI%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKGTN6OF32ZSSL3NX42GQ23VJKFK3ANCNFSM5VSFK4SA&data=05%7C01%7Ctimmcmic%40microsoft.com%7C0e66232a70d54da2f57708da32a26aa7%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637877970406561835%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iYiMggnn7Q0K%2FvrNItYxqp%2F5RdQSi5ZFaLG3QV05Qtk%3D&reserved=0. You are receiving this because you are subscribed to this thread.Message ID: @.**@.>>

MassimoPascucci commented 2 years ago

Agreed.