search

OfficeDev / ews-managed-api

Other
585 stars 321 forks source link

EWS FindItem fails on complex queries #241

Open vputsenko opened 4 years ago

vputsenko commented 4 years ago

Nature of issue

EWS FindItem request with complexity above certain started to fail

Date of first failure: 1/16/2020, 7:04:57.639 AM UTC

Failing servers: Office 365, supposedly version 15.20.2644.21, Exchange 2010-2016 do not manifest this issue

How it fails

FindItem EWS request errors with following message:

Error: ErrorInternalServerError

MessageText: An internal server error occurred. The operation failed., This query contains too many sort columns. This provider does not support more than 6 columns for a normal query, or 4 sort columns for a grouped query. Parameter name: sortColumns

Sample failing request

<Trace Tag="EwsRequestHttpHeaders" Tid="49" Time="2020-01-16 13:30:09Z">
POST /EWS/Exchange.asmx HTTP/1.1
Content-Type: text/xml; charset=utf-8
Accept: text/xml
User-Agent: ExchangeServicesClient/0.0.0.0
Accept-Encoding: gzip,deflate
client-request-id: 5102bcc1-5ef1-40c5-ad35-0553901e25b8
return-client-request-id: true
X-AnchorMailbox: *** redacted ***

</Trace>

<Trace Tag="EwsRequest" Tid="49" Time="2020-01-16 13:30:09Z" Version="0.0.0.0">
  <?xml version="1.0" encoding="utf-8"?>
  <soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
    <soap:Header>
      <t:RequestServerVersion Version="Exchange2010_SP2" />
      <t:ExchangeImpersonation>
        <t:ConnectingSID>
          <t:SmtpAddress>*** redacted ***</t:SmtpAddress>
        </t:ConnectingSID>
      </t:ExchangeImpersonation>
    </soap:Header>
    <soap:Body>
      <m:FindItem Traversal="Shallow">
        <m:ItemShape>
          <t:BaseShape>IdOnly</t:BaseShape>
          <t:BodyType>Text</t:BodyType>
          <t:AdditionalProperties>
            <t:FieldURI FieldURI="item:Subject" />
            <t:FieldURI FieldURI="calendar:Start" />
            <t:FieldURI FieldURI="calendar:End" />
            <t:FieldURI FieldURI="calendar:AppointmentState" />
            <t:ExtendedFieldURI DistinguishedPropertySetId="Meeting" PropertyId="3" PropertyType="Binary" />
            <t:FieldURI FieldURI="item:ItemId" />
            <t:FieldURI FieldURI="item:Categories" />
            <t:FieldURI FieldURI="item:ItemClass" />
            <t:FieldURI FieldURI="calendar:IsMeeting" />
          </t:AdditionalProperties>
        </m:ItemShape>
        <m:IndexedPageItemView MaxEntriesReturned="100" Offset="0" BasePoint="Beginning" />
        <m:Restriction>
          <t:And>
            <t:Or>
              <t:IsEqualTo>
                <t:ExtendedFieldURI DistinguishedPropertySetId="Meeting" PropertyId="3" PropertyType="Binary" />
                <t:FieldURIOrConstant>
                  <t:Constant Value="BAAAAIIA4AB0xbcQGoLgCAAAAADA1P970nLVAQAAAAAAAAAAEAAAALoKbuHZ9yFFixiF/WD9c/M=" />
                </t:FieldURIOrConstant>
              </t:IsEqualTo>
              <t:And>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="item:Subject" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="*** redacted ***" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="calendar:Start" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="2019-10-17T14:00:00.000Z" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="calendar:End" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="2019-10-17T14:30:00.000Z" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:Not>
                  <t:Excludes>
                    <t:FieldURI FieldURI="calendar:AppointmentState" />
                    <t:Bitmask Value="2" />
                  </t:Excludes>
                </t:Not>
              </t:And>
              <t:IsEqualTo>
                <t:ExtendedFieldURI DistinguishedPropertySetId="Meeting" PropertyId="3" PropertyType="Binary" />
                <t:FieldURIOrConstant>
                  <t:Constant Value="BAAAAIIA4AB0xbcQGoLgCAAAAADQF0k4CMbVAQAAAAAAAAAAEAAAAH51y6gpwXZGsUwXBw8l4bI=" />
                </t:FieldURIOrConstant>
              </t:IsEqualTo>
              <t:And>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="item:Subject" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="*** redacted ***" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="calendar:Start" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="2020-02-07T15:00:00.000Z" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:IsEqualTo>
                  <t:FieldURI FieldURI="calendar:End" />
                  <t:FieldURIOrConstant>
                    <t:Constant Value="2020-02-07T15:30:00.000Z" />
                  </t:FieldURIOrConstant>
                </t:IsEqualTo>
                <t:Not>
                  <t:Excludes>
                    <t:FieldURI FieldURI="calendar:AppointmentState" />
                    <t:Bitmask Value="2" />
                  </t:Excludes>
                </t:Not>
              </t:And>
            </t:Or>
            <t:Excludes>
              <t:FieldURI FieldURI="calendar:AppointmentState" />
              <t:Bitmask Value="4" />
            </t:Excludes>
            <t:Or>
              <t:Not>
                <t:Exists>
                  <t:ExtendedFieldURI DistinguishedPropertySetId="PublicStrings" PropertyName="SyncType_{E706B428-D833-44fc-97B7-8F5A8DD59D35}" PropertyType="String" />
                </t:Exists>
              </t:Not>
              <t:IsEqualTo>
                <t:ExtendedFieldURI DistinguishedPropertySetId="PublicStrings" PropertyName="SyncType_{E706B428-D833-44fc-97B7-8F5A8DD59D35}" PropertyType="String" />
                <t:FieldURIOrConstant>
                  <t:Constant Value="Event" />
                </t:FieldURIOrConstant>
              </t:IsEqualTo>
            </t:Or>
          </t:And>
        </m:Restriction>
        <m:ParentFolderIds>
          <t:FolderId Id="AAMkADNlMDJhZDY3LWUyNTUtNDBkMC1hODk4LTllYTBkZGY5YTM3MAAuAAAAAAC4YZOICYAzQJ+HEQvGO9soAQBbhZs7tAQ5R5IfrJ0uQp8wAAAAPeC0AAA=" ChangeKey="AgAAABYAAAADvS72GAPGRZ12mQnXam3oAADm5Lgn" />
        </m:ParentFolderIds>
      </m:FindItem>
    </soap:Body>
  </soap:Envelope>
</Trace>

<Trace Tag="EwsResponseHttpHeaders" Tid="49" Time="2020-01-16 13:30:10Z">
HTTP/1.1 200 OK
Transfer-Encoding: chunked
Content-Encoding: gzip
Vary: Accept-Encoding
request-id: b9456490-7fd4-4f61-9c79-010ddf73ecae
client-request-id: 5102bcc1-5ef1-40c5-ad35-0553901e25b8
X-CalculatedFETarget: BN7PR02CU001.internal.outlook.com
X-BackEndHttpStatus: 200,200
X-FEProxyInfo: BN7PR02CA0006.NAMPRD02.PROD.OUTLOOK.COM
X-CalculatedBETarget: BN8PR10MB4084.namprd10.prod.outlook.com
X-RUM-Validated: 1
X-MailboxGuid: 3e02ad67-e255-40d0-a898-9ea0ddf9a370
x-EwsHandler: FindItem
X-BeSku: WCS5
X-DiagInfo: BN8PR10MB4084
X-BEServer: BN8PR10MB4084
X-Proxy-RoutingCorrectness: 1
X-Proxy-BackendServerStatus: 200
X-FEServer: BN7PR02CA0006,MN2PR05CA0045
Cache-Control: private
Content-Type: text/xml; charset=utf-8
Date: Thu, 16 Jan 2020 13:30:10 GMT
Set-Cookie: exchangecookie=f06a9fdd9a8646299b768b52b5cc6518; path=/; secure
Server: Microsoft-IIS/10.0
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET

</Trace>

<Trace Tag="EwsResponse" Tid="49" Time="2020-01-16 13:30:10Z" Version="0.0.0.0">
  <?xml version="1.0" encoding="utf-8"?>
  <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/">
    <s:Header>
      <h:ServerVersionInfo MajorVersion="15" MinorVersion="20" MajorBuildNumber="2644" MinorBuildNumber="21" Version="V2018_01_08" xmlns:h="http://schemas.microsoft.com/exchange/services/2006/types" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" />
    </s:Header>
    <s:Body>
      <m:FindItemResponse xmlns:m="http://schemas.microsoft.com/exchange/services/2006/messages" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
        <m:ResponseMessages>
          <m:FindItemResponseMessage ResponseClass="Error">
            <m:MessageText>An internal server error occurred. The operation failed., This query contains too many sort columns. This provider does not support more than 6 columns for a normal query, or 4 sort columns for a grouped query.
Parameter name: sortColumns</m:MessageText>
            <m:ResponseCode>ErrorInternalServerError</m:ResponseCode>
            <m:DescriptiveLinkKey>0</m:DescriptiveLinkKey>
          </m:FindItemResponseMessage>
        </m:ResponseMessages>
      </m:FindItemResponse>
    </s:Body>
  </s:Envelope>
</Trace>

Preliminary investigation

Severity and Impact Thousands of production users are affected.

Attempts to isolate Issue started to manifest itself on Jan 16 without any changes on application side, so update or change to Office 365 Exchange Online is a suspect.

Scope

Office 365 ONLY. Exchange 2010-2016 do not have this issue.

Office 365: Only part of the customers are affected, while other part of customers are working OK. Possibly because of staged update of Office 365 instances

Issue definitely seen on Office 365 Exchange version: 15.20.2644.21

Issue seem to not reproduce itself on: 15.20.2644.20

Sample Exchange servers where issue seen:

X-BEServer: BN8PR10MB4084 X-BEServer: BYAPR13MB2583

Narrowing down problem

Issue seem to be related with query complexity. As soon as we drop amount of fields in restriction to be less or equal than 5, query starts to work fine.

Is there any known fix/workaround to the issue?