ResponseCompression async APIs write synchronously

[Tratcher]

This is relevant because the servers are planning to disable sync IO: https://github.com/aspnet/HttpSysServer/issues/366

net461

Server stack trace:
   at Microsoft.AspNetCore.Server.HttpSys.ResponseBody.Write(Byte[] buffer, Int32 offset, Int32 count)
   at Microsoft.AspNetCore.Server.HttpSys.ResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Compression.DeflateStream.DoMaintenance(Byte[] array, Int32 offset, Int32 count)
   at System.IO.Compression.DeflateStream.InternalWrite(Byte[] array, Int32 offset, Int32 count, Boolean isAsync)
   at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)
   at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

Exception rethrown at [0]:
   at System.IO.Compression.DeflateStream.EndWrite(IAsyncResult asyncResult)
   at System.IO.Compression.GZipStream.EndWrite(IAsyncResult asyncResult)
   at System.IO.Stream.<>c.<BeginEndWriteAsync>b__53_1(Stream stream, IAsyncResult asyncResult)
   at System.Threading.Tasks.TaskFactory`1.FromAsyncTrimPromise`1.Complete(TInstance thisRef, Func`3 endMethod, IAsyncResult asyncResult, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.AspNetCore.ResponseCompression.BodyWrapperStream.<WriteAsync>d__32.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at ResponseCompressionSample.Startup.<>c.<<Configure>b__1_4>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.AspNetCore.Server.HttpSys.MessagePump.<ProcessRequestAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.AspNetCore.Server.HttpSys.MessagePump.<ProcessRequestAsync>d__20.MoveNext()

netcoreapp2.0

System.InvalidOperationException: Syncronous IO APIs are disabled, see AllowSynchronousIO.
   at Microsoft.AspNetCore.Server.HttpSys.ResponseBody.Write(Byte[] buffer, Int32 offset, Int32 count)
   at Microsoft.AspNetCore.Server.HttpSys.ResponseStream.Write(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.Compression.DeflateStream.PurgeBuffers(Boolean disposing)
   at System.IO.Compression.DeflateStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at System.IO.Compression.GZipStream.Dispose(Boolean disposing)
   at System.IO.Stream.Close()
   at Microsoft.AspNetCore.ResponseCompression.BodyWrapperStream.Dispose(Boolean disposing) in C:\git\Universe\BasicMiddleware\src\Microsoft.AspNetCore.ResponseCompression\BodyWrapperStream.cs:line 46
   at System.IO.Stream.Close()
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.<Invoke>d__3.MoveNext() in C:\git\Universe\BasicMiddleware\src\Microsoft.AspNetCore.ResponseCompression\ResponseCompressionMiddleware.cs:line 74
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Server.HttpSys.MessagePump.<ProcessRequestAsync>d__20.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Microsoft.AspNetCore.Server.HttpSys.MessagePump.<ProcessRequestAsync>d__20.MoveNext()

[halter73]

I think we should open an issue in https://github.com/dotnet/corefx for DeflateStream. @Tratcher Did you verify that DeflateStream.FlushAsync() was being called by ResponseCompression prior to DeflateStream.Dispose()?

[Tratcher]

No it isn't. Easy enough to add: https://github.com/aspnet/BasicMiddleware/blob/dev/src/Microsoft.AspNetCore.ResponseCompression/ResponseCompressionMiddleware.cs#L73

[halter73]

We should check tomorrow see if that fixes this issue on .NET Core. It's not unreasonable for DeflateStream.Dispose() to call InnerStream.Write() if there's still unflushed output.

[Tratcher]

No, I had tried it, it was still calling Flush.

[muratg]

Bringing back to triage

[Zoxive]

This is still missing in 2.1.0-rc1-final. Will this be done in RC2?

[Tratcher]

No this isn't getting fixed for 2.1. We need to get some work done on the underlying compression streams to enable it. The worst cases I saw when I checked last were Flush and Dispose.

[muratg]

cc @davidfowl Something you may want to call out in your amazing new 'diagnosability' doc :)

[muratg]

We will react to https://github.com/dotnet/corefx/issues/32665

[Tratcher]

This is now unblocked by https://github.com/dotnet/corefx/pull/33415, pending picking up a new build.