EWS Throttling issues being rolled out worldwide on 365

[vlw007]

Starting April 6, we are seeing EWS throttling issues when there is barely any EWS activity on a tenant.

The error looks like this: [You have exceeded the available concurrent connections for your account. Try again once your other requests have completed.] ErrorDetails; Key=[Policy], Value=[MaxConcurrency] ErrorDetails; Key=[MaxConcurrencyLimit], Value=[27] ErrorDetails; Key=[ErrorMessage], Value=[This operation exceeds the throttling budget for policy part 'MaxConcurrency', policy value '27', Budget type: 'Ews'. Suggested backoff time 0 ms.]

When this step executes, Our SaaS platform has only a single thread (no concurrency) and our backoff time is way larger than zero ms.

We made the request to remove throttling policies for 90 days. The support request suggested that we wait 15 minutes for the policy to be updated

We waited two hours and there does not appear to be any difference. The same error appears. Another application using EWS has also started having problems at almost the exact same time.

Does anyone notice this happening too?

[cmcveen]

Yes, I have noticed this same issue over the past couple of days in a few tenants.

[mariusan]

Same here on a couple of tenants

Regards Marius

[joshmadams]

I am seeing this error in our tenant as well.

[PhilbyTheGreat]

We are seeing this in our tenant too.

[mariusan]

Below is the result of a trace. This is in response to the very first call and there are no other uses of the account.

<Trace Tag="EwsResponse" Tid="5" Time="2023-04-13 06:00:12Z" Version="[2.2.1.0](http://2.2.1.0/)">
  <?xml version="1.0" encoding="utf-8"?>
  <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Body>
      <s:Fault>
        <faultcode xmlns:a="http://schemas.microsoft.com/exchange/services/2006/types">a:ErrorExceededConnectionCount</faultcode>
        <faultstring xml:lang="en-US">You have exceeded the available concurrent connections for your account.  Try again once your other requests have completed.</faultstring>
        <detail>
          <e:ResponseCode xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">ErrorExceededConnectionCount</e:ResponseCode>
          <e:Message xmlns:e="http://schemas.microsoft.com/exchange/services/2006/errors">You have exceeded the available concurrent connections for your account.  Try again once your other requests have completed.</e:Message>
          <t:MessageXml xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
            <t:Value Name="Policy">MaxConcurrency</t:Value>
            <t:Value Name="MaxConcurrencyLimit">27</t:Value>
            <t:Value Name="ErrorMessage">This operation exceeds the throttling budget for policy part 'MaxConcurrency', policy value '27',  Budget type: 'Ews'.  Suggested backoff time 0 ms.</t:Value>
          </t:MessageXml>
        </detail>
      </s:Fault>
    </s:Body>
  </s:Envelope>
</Trace>

[halliballi123]

same here, problem started yesterday evening.

[joshmadams]

This issue started around 7am EST yesterday (4/12/23) for us.

[Yup-maker]

5 customers also have this problem with Public Folders. (However, I myself can not reproduce it)

This is my Exchange Online Version: 15.20.6277.038 V2018_01_08
This is one of my customers: 15.20.6298.028 V2018_01_08

[halliballi123]

5 customers also have this problem with Public Folders. (However, I myself can not reproduce it)

This is my Exchange Online Version: 15.20.6277.038 V2018_01_08 This is one of my customers: 15.20.6298.028 V2018_01_08

In our case it's also public folders. We create contacts in public folders. Version is 15.20.6298.30

[mariusan]

Might be the same issue: https://admin.microsoft.com/Adminportal/Home#/servicehealth/:/alerts/EX540990

[vlw007]

Are you able to get the Exchange Online Version via an API call ?

---

From: halliballi123 @.> Sent: Friday, April 14, 2023 6:11:58 AM To: OfficeDev/ews-managed-api @.> Cc: Vernon L. Weitzman (Cira Apps) @.>; Author @.> Subject: Re: [OfficeDev/ews-managed-api] EWS Throttling issues being rolled out worldwide on 365 (Issue #293)

5 customers also have this problem with Public Folders. (However, I myself can not reproduce it)

This is my Exchange Online Version: 15.20.6277.038 V2018_01_08 This is one of my customers: 15.20.6298.028 V2018_01_08

In our case it's also public folders. We create contacts in public folders.

— Reply to this email directly, view it on GitHubhttps://github.com/OfficeDev/ews-managed-api/issues/293#issuecomment-1507974268, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJDIBQRPNUFYCARDXKYCJ63XBDTC5ANCNFSM6AAAAAAW4GA22A. You are receiving this because you authored the thread.Message ID: @.***>

[Yup-maker]

Yes by using ExchangeServerInfo: service.ServerInfo?.ToString()

Are you able to get the Exchange Online Version via an API call ? … ____ From: halliballi123 @.> Sent: Friday, April 14, 2023 6:11:58 AM To: OfficeDev/ews-managed-api @.> Cc: Vernon L. Weitzman (Cira Apps) @.>; Author @.> Subject: Re: [OfficeDev/ews-managed-api] EWS Throttling issues being rolled out worldwide on 365 (Issue #293) 5 customers also have this problem with Public Folders. (However, I myself can not reproduce it) This is my Exchange Online Version: 15.20.6277.038 V2018_01_08 This is one of my customers: 15.20.6298.028 V2018_01_08 In our case it's also public folders. We create contacts in public folders. — Reply to this email directly, view it on GitHub<#293 (comment)>, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJDIBQRPNUFYCARDXKYCJ63XBDTC5ANCNFSM6AAAAAAW4GA22A. You are receiving this because you authored the thread.Message ID: @.***>

[vlw007]

Are you able to get the Exchange Online Version via an API call ?

---

From: halliballi123 @.**@.>> Sent: Friday, April 14, 2023 6:11:58 AM To: OfficeDev/ews-managed-api @.**@.>> Cc: Vernon L. Weitzman (Cira Apps) @.**@.>>; Author @.**@.>> Subject: Re: [OfficeDev/ews-managed-api] EWS Throttling issues being rolled out worldwide on 365 (Issue #293)

5 customers also have this problem with Public Folders. (However, I myself can not reproduce it)

This is my Exchange Online Version: 15.20.6277.038 V2018_01_08 This is one of my customers: 15.20.6298.028 V2018_01_08

In our case it's also public folders. We create contacts in public folders.

- Reply to this email directly, view it on GitHubhttps://github.com/OfficeDev/ews-managed-api/issues/293#issuecomment-1507974268, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJDIBQRPNUFYCARDXKYCJ63XBDTC5ANCNFSM6AAAAAAW4GA22A. You are receiving this because you authored the thread.Message ID: @.***>

[ToS0]

Same here - since yesterday

[viebrix]

We also had the issue with public folder, but it seems it is right now solved

[Andy1860]

Same here, seems to be solved.

[viebrix]

Greg Taylor from the Exchange Teams asked here why and how people use public folders: techcommunity - exchange-team-blog I think this is a great chance to help the team to improve exchange for us.

[halliballi123]

problem still there

[viebrix]

@halliballi123 I'm only user and not experience in this topic. But did you try to request to remove throttling policies for 90 days, as vlw007 is describing in his first post? We also did this step last week. Maybe this makes a difference?

[halliballi123]

I don't know how and where to put that request

[viebrix]

Found this description in the web: Sign in to Microsoft 365 Admin Center and click on the Help icon in the top corner of the screen. Search for EWS and select the Increase EWS Throttling Policy article that is suggested.In the Run diagnostics section that is displayed, select Run Tests. If the diagnostic returns a The Exchange Web Services are throttled message, you have the option to change (ease) the policy to allow your migration to complete. Choose the Duration (Days) that you want to relax the policy for from the drop down menu. You can choose between 20, 60 and 90 days. Place a tick in the checkbox to acknowledge that want to make the specified changes. Select Update Settings to apply the changes to the policy. Changes can take up to 30 minutes to take effect. (source of text, support.cloudm.io)

[Andy1860]

Found this description in the web: Sign in to Microsoft 365 Admin Center and click on the Help icon in the top corner of the screen. Search for EWS and select the Increase EWS Throttling Policy article that is suggested.In the Run diagnostics section that is displayed, select Run Tests. If the diagnostic returns a The Exchange Web Services are throttled message, you have the option to change (ease) the policy to allow your migration to complete. Choose the Duration (Days) that you want to relax the policy for from the drop down menu. You can choose between 20, 60 and 90 days. Place a tick in the checkbox to acknowledge that want to make the specified changes. Select Update Settings to apply the changes to the policy. Changes can take up to 30 minutes to take effect. (source of text, support.cloudm.io)

I already tried this last week when the problems started with no improvement.

[Andy1860]

I think you just have to wait until it works again. We haven't had any more problems since yesterday afternoon.

Last update in M365 Health Center:

April 17, 2023 7:24 PM Title: Some users may be unable to access, view, add, or select public folders and calendars in Outlook on the web User impact: Users may be unable to access, view, add, or select public folders and calendars in Outlook on the web. Current status: We've determined the status of the fix deployment to currently stand at 80 percent saturation, and we’ll provide an estimated timeline for resolution once available. Scope of impact: Some users attempting to perform multiple public folder or calendar related actions in Outlook on the web may be impacted. Start time: Tuesday, April 11, 2023, 1:37 PM (11:37 AM UTC) Root cause: A recent service update contained a code issue that’s preventing users from accessing, viewing, adding, or selecting public folders and calendars in Outlook on the web. Next update by: Tuesday, April 18, 2023, 8:00 PM (6:00 PM UTC)

[halliballi123]

I think you just have to wait until it works again. We haven't had any more problems since yesterday afternoon.

Last update in M365 Health Center:

April 17, 2023 7:24 PM Title: Some users may be unable to access, view, add, or select public folders and calendars in Outlook on the web User impact: Users may be unable to access, view, add, or select public folders and calendars in Outlook on the web. Current status: We've determined the status of the fix deployment to currently stand at 80 percent saturation, and we’ll provide an estimated timeline for resolution once available. Scope of impact: Some users attempting to perform multiple public folder or calendar related actions in Outlook on the web may be impacted. Start time: Tuesday, April 11, 2023, 1:37 PM (11:37 AM UTC) Root cause: A recent service update contained a code issue that’s preventing users from accessing, viewing, adding, or selecting public folders and calendars in Outlook on the web. Next update by: Tuesday, April 18, 2023, 8:00 PM (6:00 PM UTC)

thanks, we tried the fix but still no improvement, so maybe you are right. We have to wait ...

[halliballi123]

finally it is working again ...