search

timmcmic / DLConversionV2

MIT License
44 stars 9 forks source link

Multiple Migrations are using 5 jobs, but EXO PS only allows 3 concurrent sessions #55

Closed PeterNagl closed 2 years ago

PeterNagl commented 2 years ago

Hi Tim,

if I use "Start-MultipleDistributionListMigration" the hardcoded(?) ThreadCount is 5, however Exchange Online Powershell is limiting to 3 concurrent session, which fails 2 of the 5 jobs.

I found this in your code: [int]$maxThreadCount = 5

Begin processing batches of members in the SMTP array.

Current max jobs recommended 5 per batch.

But no parameter to change this default behavior. Some people point out that one can request more concurrent sessions from Microsoft Support. https://blog.quadrotech-it.com/blog/office-365-exchange-online-powershell-throttling/

Is there a way different way to specify the ThreadCount? If not, maybe you can change the default to match the default PowerShellMaxConcurrency or add a parameter.

That would be great. kind regards Peter

Addition error details: [6/23/2022 3:51:27 PM] - BEGIN NEW-EXCHANGEONLINEPOWERSHELLSESSION [6/23/2022 3:51:27 PM] - **** [6/23/2022 3:51:27 PM] - ExchangeOnlineCredentialsUserName = admin@xxx.onmicrosoft.com [6/23/2022 3:51:27 PM] - Is certificate auth = False [6/23/2022 3:51:27 PM] - ExchangeOnlineCommandPrefix = O365 [6/23/2022 3:51:27 PM] - Creating the exchange online powershell session. New-ExoPSSession : Processing data from remote server outlook.office365.com failed with the following error message: [AuthZRequestId=xxx][FailureCategory=AuthZ-AuthorizationException] Fail to create a runspace because you have exceeded the maximum number of connections allowed : 3 for the policy party : MaxConcurrency. Please close existing runspace and try again. Policy: CN=GlobalThrottlingPolicy_xxx,CN=Global Settings,CN=ExchangeLabs,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=eurprd09,DC=prod,DC=outlook,DC=com; Snapshot: Owner: Sid~xxx~WSMan~false BudgetType: WSMan ActiveRunspaces: 3/3 Balance: 600000/1800000/-3000000 PowerShellCmdletsLeft: 200/200 ExchangeCmdletsLeft: 9223372036854775807/Unlimited CmdletTimePeriod: 5 DestructiveCmdletsLeft: 9223372036854775807/Unlimited DestructiveCmdletTimePeriod: Unlimited QueueDepth: Unlimited MaxRunspacesTimePeriod: 60 RunSpacesRemaining: 2/5 LastTimeFrameUpdate: 6/23/2022 1:51:25 PM LastTimeFrameUpdateDestructiveCmdlets: 6/23/2022 1:51:25 PM LastTimeFrameUpdateMaxRunspaces: 6/23/2022 1:51:25 PM Locked: False LockRemaining: 00:00:00 For more information, see the about_Remote_Troubleshooting Help topic. At C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\2.0.5\netFramework\ExchangeOnlineManagement.psm1:475 char:30

timmcmic commented 2 years ago

Peter - I neglected to comment on this one. Did you ever open a support case up to get your powershell throttling limits reset. If not if you let me know the onmicrosoft.com tenant name at the email address I provided I can help with that. You should be able to do more than 3.

PeterNagl commented 2 years ago

Yes, I opened a ticket for the O365 support. That was the answer:

Issue description: Increase Exchange Online PowerShell connection limit

_Resolution Steps I would like to inform you that I have increased the throttling limit of your tenant for the next 52 days

Please note that even with the increment EWS imports will still be limited to 150mb per 5 minutes per mailbox to achieve higher migration throughput speeds, please migrate more users concurrently._

I didn´t have the possibility to test if we now have 5 concurrent connections. Also I wonder why the Support Engineer is talking EWS Limits, but maybe they enabled a throttling policy that affects more services.

Also it would be great to have that increase permanently for a 10k+ Mailboxes company. I will send you the tenant id via mail.

PeterNagl commented 2 years ago

I close this because the tenant has now more that 3 connections. But still the default should match the EXO default :)

timmcmic commented 1 year ago

Raise a case and have them adjust the powershell throttling policy. You should easily be able to do 5.

Tim

============================== Timothy J. McMichael Senior Support Escalation Engineer @.**@.> (980)-776-7465

Hours: Sunday - Wednesday 08:00 - 16:00 eastern time zone.

Manager: Tom Roughley @.**@.>)

Premier Support - (800)-936-3100 Broad Commercial Support - (800)-936-4900

==============================

From: Peter Nagl @.> Sent: Thursday, June 23, 2022 10:28 AM To: timmcmic/DLConversionV2 @.> Cc: Subscribed @.***> Subject: [timmcmic/DLConversionV2] Multiple Migrations are using 5 jobs, but EXO PS only allows 3 concurrent sessions (Issue #55)

Hi Tim,

if I use "Start-MultipleDistributionListMigration" the hardcoded(?) ThreadCount is 5, however Exchange Online Powershell is limiting to 3 concurrent session, which fails 2 of the 5 jobs.

I found this in your code: [int]$maxThreadCount = 5

Begin processing batches of members in the SMTP array.

Current max jobs recommended 5 per batch.

But no parameter to change this default behavior. Some people point out that one can request more concurrent sessions from Microsoft Support. https://blog.quadrotech-it.com/blog/office-365-exchange-online-powershell-throttling/https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.quadrotech-it.com%2Fblog%2Foffice-365-exchange-online-powershell-throttling%2F&data=05%7C01%7Ctimmcmic%40microsoft.com%7C4a0d0314335e4467d85408da55249049%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637915912786065276%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=yYUlNegstTnRX5w%2FmOpwLUi1aSF4G%2FYfMolg4q3%2F0gg%3D&reserved=0

Is there a way different way to specify the ThreadCount? If not, maybe you can change the default to match the default PowerShellMaxConcurrency or add a parameter.

That would be great. kind regards Peter

Addition error details: [6/23/2022 3:51:27 PM] - BEGIN NEW-EXCHANGEONLINEPOWERSHELLSESSION [6/23/2022 3:51:27 PM] - **** [6/23/2022 3:51:27 PM] - ExchangeOnlineCredentialsUserName = @.**@.> [6/23/2022 3:51:27 PM] - Is certificate auth = False [6/23/2022 3:51:27 PM] - ExchangeOnlineCommandPrefix = O365 [6/23/2022 3:51:27 PM] - Creating the exchange online powershell session. New-ExoPSSession : Processing data from remote server outlook.office365.com failed with the following error message: [AuthZRequestId=xxx][FailureCategory=AuthZ-AuthorizationException] Fail to create a runspace because you have exceeded the maximum number of connections allowed : 3 for the policy party : MaxConcurrency. Please close existing runspace and try again. Policy: CN=GlobalThrottlingPolicy_xxx,CN=Global Settings,CN=ExchangeLabs,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=eurprd09,DC=prod,DC=outlook,DC=com; Snapshot: Owner: SidxxxWSMan~false BudgetType: WSMan ActiveRunspaces: 3/3 Balance: 600000/1800000/-3000000 PowerShellCmdletsLeft: 200/200 ExchangeCmdletsLeft: 9223372036854775807/Unlimited CmdletTimePeriod: 5 DestructiveCmdletsLeft: 9223372036854775807/Unlimited DestructiveCmdletTimePeriod: Unlimited QueueDepth: Unlimited MaxRunspacesTimePeriod: 60 RunSpacesRemaining: 2/5 LastTimeFrameUpdate: 6/23/2022 1:51:25 PM LastTimeFrameUpdateDestructiveCmdlets: 6/23/2022 1:51:25 PM LastTimeFrameUpdateMaxRunspaces: 6/23/2022 1:51:25 PM Locked: False LockRemaining: 00:00:00 For more information, see the about_Remote_Troubleshooting Help topic. At C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\2.0.5\netFramework\ExchangeOnlineManagement.psm1:475 char:30