"Too many concurrent connections opened" errors with OAuth

[viktdm]

Hi,

We have an application using EWS Managed API and connecting to Microsoft Exchange Online (Office 365) via EWS with OAuth. Recently, we’ve started observing a lot of "Too many concurrent connections opened" exceptions thrown when our app is retrieving messages from Exchange. Even if we limit the number of parallel processing tasks to one and there are no other clients accessing a mailbox, the exception still comes out after a few EWS requests sent one after another. The problem seems to have something to do with OAuth. If we use the basic authentication with special service account credentials, everything works fine.

It is easy to reproduce the problem by making several requests to get an item by id. For example, start making POST requests with a similar body several times one after another. Use OAuth, so provide Authorization header with a token:

<?xml version="1.0" encoding="utf-8"?>
<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
               xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" 
               xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" 
               xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
  <soap:Header>
    <t:RequestServerVersion Version="Exchange2016" />
    <t:ExchangeImpersonation>
        <t:ConnectingSID>
          <t:SmtpAddress>test@test.onmicrosoft.com</t:SmtpAddress>
        </t:ConnectingSID>
    </t:ExchangeImpersonation>
  </soap:Header>
  <soap:Body>
    <m:GetItem>
      <m:ItemShape>
        <t:BaseShape>IdOnly</t:BaseShape>
        <t:AdditionalProperties>
          <t:FieldURI FieldURI="item:Subject" />
        </t:AdditionalProperties>
      </m:ItemShape>
      <m:ItemIds>
        <t:ItemId Id="AAMkAGQ5M..." />
      </m:ItemIds>
    </m:GetItem>
  </soap:Body>
</soap:Envelope>

Several requests will be successful, but after about 10 requests EWS would start returning "Too many concurrent connections opened" error. With Basic Authentication the same request could be made 1000 times, with 20 request in parallel without a problem.

BTW, it’s worth mentioning that it takes quite long to get responses from EWS with OAuth comparing with the basic authentication. While with the basic authentication, it may take around 70 ms to get an item by its Id, with OAuth it takes from 300 to 1300 ms.

Any help to resolve the issue will be appreciated.

[viktdm]

I wonder if the problem we experience has something to do with the vulnerability patched by Microsoft recently (see https://www.cyberark.com/threat-research-blog/blackdirect-microsoft-azure-account-takeover/). According to our logs, the problem appeared around the time when the fix was apparently applied. Could someone from Microsoft team (ping @darrelmiller) please clarify.

[viktdm]

The issue can be closed. Worked out with Microsoft support to find out that the problem was with missing X-AnchorMailbox header in EWS requests. It was still odd that everything worked just fine with Basic auth.

[FilipRychnavsky]

I have experienced the same problem. Glen's Exchange and Office 365 Dev Blog adds X-AnchorMailbox to the HTTP Headers (rows 77 - 88) during impersonification. I use following code with Microsoft.Exchange.WebServices.Data namespace of EWS managed API: if (m_rExchangeService->HttpHeaders->ContainsKey("X-AnchorMailbox")) { m_rExchangeService->HttpHeaders["X-AnchorMailbox"] = sUserToImpersonificate; } else { m_rExchangeService->HttpHeaders->Add("X-AnchorMailbox", sUserToImpersonificate); m_rExchangeService->HttpHeaders->Add("X-PreferServerAffinity", "true"); } Debug::WriteLine(System::String::Format("AnchorMailbox: {0}", m_rExchangeService->HttpHeaders["X-AnchorMailbox"]));

[harry-flw]

I have tried to use X-AnchorMailbox - it doesn't help.

I have experienced the same problem. And I have no solution yet.

I have multithreading application which synchronizes MS SharePoint data about organizations with business contacts in user's folders of MS Exchange.

It works fine when I use 1 or 2 threads (1 or 2 user's folders). If number of threads more than 2 I constantly have an error "Too many concurrent connections opened."

I do POST requests duly completed as SOAP and use IHttpClientFactory to create a HttpClient.

I always dispose every HttpRequestMessage and HttpClient.

In my case I don't use impersonification because I have special rights to mailboxes. Moreover impersonification is not configured.

Absolutely the same issue with EWS Managed API.