Open DevilGeek opened 5 years ago
Can you provide a requestId (from the HTTP response headers)?
We've been experiencing the same since roughly Tuesday. Here's an example requestid:
request-id: 9a02889a-cccc-45f8-b497-e7fb3f7bb25c
I'm working with a customer that has been experiencing this since Wednesday. Here is a sample request-id from today: 3e5ce14e-cdfd-4793-b4f2-473905a9c073
Can you provide a requestId (from the HTTP response headers)?
request-id: c501a35c-27ed-45f0-bf64-fc7e2880d10a
<Trace Tag="EwsResponseHttpHeaders" Tid="32" Time="2018-10-29 10:27:39Z">
HTTP/1.1 500 Internal Server Error
Cache-Control: private
Content-Length: 783
Content-Type: text/xml; charset=utf-8
Server: Microsoft-IIS/10.0
request-id: c501a35c-27ed-45f0-bf64-fc7e2880d10a
X-CalculatedFETarget: PR2P264CU001.internal.outlook.com
X-BackEndHttpStatus: 500,500
Thanks - I forgot to mention that I need the UTC time of the requests as well. @DevilGeek provided one - just waiting for the logs to be uploaded to the data repo so I can query for it. To get the info more quickly, if you can also provide for me the X-BEServer response header, I could grab the logs from the server itself.
@davster From the same response
X-FEProxyInfo: PR2P264CA0011.FRAP264.PROD.OUTLOOK.COM
X-CalculatedBETarget: PR2P264MB0894.FRAP264.PROD.OUTLOOK.COM
X-RUM-Validated: 1
X-AspNet-Version: 4.0.30319
X-BeSku: WCS5
X-DiagInfo: PR2P264MB0894
X-BEServer: PR2P264MB0894
X-FEServer: PR2P264CA0011,LO2P265CA0131
X-Powered-By: ASP.NET
@DevilGeek - thanks for the info. I found the log and it appears that you are hitting a null ref on our side (a regression). I have forwarded the info to the calendaring team to fix.
@mattaustin83 and @sgreenlee11 - if you could also provide a UTC date time with your request ids and an X-BEServer, I can confirm if you are seeing the same thing or not.
@davster
Here is that info:
HTTP/1.1 500 Internal Server Error Cache-Control: private Content-Length: 783 Content-Type: text/xml; charset=utf-8 Server: Microsoft-IIS/10.0 request-id: 3e5ce14e-cdfd-4793-b4f2-473905a9c073 X-CalculatedBETarget: DM5PR0701MB3813.namprd07.prod.outlook.com X-BackEndHttpStatus: 500 X-RUM-Validated: 1 X-AspNet-Version: 4.0.30319 X-BeSku: WCS5 X-DiagInfo: DM5PR0701MB3813 X-BEServer: DM5PR0701MB3813 X-Powered-By: ASP.NET X-FEServer: DM5PR1501CA0024 Date: Fri, 26 Oct 2018 18:59:47 GMT
Thanks!
@sgreenlee11 - yep, you are hitting the same issue as @DevilGeek. The calendaring folks should be in here shortly today. I already opened a bug and assigned it to the dev in question. Looking at the change, it should be a trivial fix on our side.
@davster here's more info from my end:
[X-CalculatedFETarget, BN6PR04CU002.internal.outlook.com]
[X-BackEndHttpStatus, 500,500]
[X-FEProxyInfo, BN6PR04CA0053.NAMPRD04.PROD.OUTLOOK.COM]
[X-CalculatedBETarget, BN6PR16MB1683.namprd16.prod.outlook.com]
[X-RUM-Validated, 1]
[X-BeSku, Gen9]
[X-DiagInfo, BN6PR16MB1683]
[X-BEServer, BN6PR16MB1683]
[X-FEServer, BN6PR04CA0053,BYAPR07CA0032]
[Content-Length, 783]
[Cache-Control, private]
[Content-Type, text/xml; charset=utf-8]
[Date, Mon, 29 Oct 2018 15:13:12 GMT]
[Set-Cookie, exchangecookie=b17bcb3dbafd4c1eb49c5efe6096f5cb; expires=Tue, 29-Oct-2019 15:13:12 GMT; path=/; HttpOnly]
[Server, Microsoft-IIS/10.0]
[X-AspNet-Version, 4.0.30319]
[X-Powered-By, ASP.NET]```
@davster great, let us know when it's fixed.
@mattaustin83 - was that for the same requestId that you have pasted above? I didn't find any logs for that one.
@davster Whoops sorry that was for a new request and it looks like I missed the request-id. Here's a new one:
[request-id, 7a3ec76a-d922-4907-ac48-90ed7f1c70b8] [X-CalculatedFETarget, MW2PR2101CU001.internal.outlook.com] [X-BackEndHttpStatus, 500,500] [Content-Length, 783] [Cache-Control, private] [Content-Type, text/xml; charset=utf-8] [Server, Microsoft-IIS/10.0] [X-FEProxyInfo, MW2PR2101CA0032.NAMPRD21.PROD.OUTLOOK.COM] [X-CalculatedBETarget, BN6PR16MB1683.namprd16.prod.outlook.com] [X-RUM-Validated, 1] [X-AspNet-Version, 4.0.30319] [X-BeSku, Gen9] [X-DiagInfo, BN6PR16MB1683] [X-BEServer, BN6PR16MB1683] [X-FEServer, MW2PR2101CA0032,BYAPR01CA0041] [X-Powered-By, ASP.NET] [Date, Mon, 29 Oct 2018 15:38:43 GMT]
@mattaustin83 - Yep same issue (null ref). Will respond back to the calendaring folks that we have multiple instances of this so we make sure there is enough urgency. Thanks for the headers.
@davster Any updates on when this might be fixed? Thanks!
It seems that if the account making the call has a mailbox or if it is impersonating a user, it does not crash.
Hmm, we are impersonating and still seeing the same exception when grabbing availability for 1 attendee. If I pass in multiple attendees no exception is thrown, but all returned AttendeeAvailability objects have an ErrorCode of ErrorProxyRequestProcessingFailed and ErrorMessage of
Proxy web request failed. , inner exception: System.Web.Services.Protocols.SoapException: An internal server error occurred. The operation failed.
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.EndInvoke(IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.Availability.Proxy.Service.EndGetUserAvailability(IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.Availability.FreeBusyApplication.EndProxyWebRequest(ProxyWebRequest proxyWebRequest, QueryList queryList, IService service, IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.Availability.ProxyWebRequest.EndInvoke(IAsyncResult asyncResult)
at Microsoft.Exchange.InfoWorker.Common.Availability.AsyncWebRequest.EndInvokeWithErrorHandling()
The null ref issue fix was checked in last week. Fixed in build 15.20.1326.000 which is currently rolling out world wide.
I am experiencing the same issue. Below is a couple of failed requests:
Everything was working fine. But then 2 weeks ago it suddenly stopped working. I have customers waiting for that to be solved. @davster, do you have any ideas when it is going to be fixed?
Hi @davster. The issue seems to be resolved now. I do not know what you did but let's hope that you did something great :)
:) I had the calendaring devs fix the issue and then I backported the fix it all the builds currently in production. So it was just a matter of waiting until it rolled out. Glad that it is fixed for you!
The "GetUserAvailability" method has been returning an internal server error for several days now. tested on 3 different Office 365 tenants.
is this a known issues ?