Getting error "System.InvalidOperationException: The stream was already consumed. It cannot be read again." when using Proxy settings to make a request.

[spati2]

I am making an Http request where the body is a stream. I am getting an error "System.InvalidOperationException: The stream was already consumed. It cannot be read again." as a response. The stream in the message body has a CanSeek value set to false. While going through the corefx code base I noticed the line of code here (https://github.com/dotnet/corefx/blob/53be85c2fe473fdea8c001e3d9fd81dd478b858e/src/System.Net.Http/src/System/Net/Http/StreamContent.cs#L88) that is responsible for the error response. I didnt encounter this issue in Net45. Why this change in behavior?

[davidsh]

cc: @stephentoub @geoffkizer

[karelz]

@spati2 which product do you see the failure on? .NET Core? Which version? Or is it .NET Framework 4.6.x/4.7.x?

[spati2]

@karelz I am running a .NetCoreApp 2.0 application.

[karelz]

OK, which OS? Windows / Linux? Which version/distro? Do you have a minimal repro which we could try locally?

[spati2]

Hi @karelz sorry about the delay. I am running my application on a windows 10 platform. I have been trying to make a PutObject request using AWSSDK.S3 dotnet package with the proxy settings on and that is encountering the said issue.

[karelz]

@spati2 are you able to reproduce it locally / without AWS SDK? We need ideally isolated repro we can easily try in-house.

[karelz]

Closing as not actionable. If there is a repro / more details, we can reopen.

[spati2]

Hi @karelz . Sorry for the delay. I have created a repro (https://github.com/spati2/CoreCLRProxyIssueRepro) for the exact issue I am facing. Please let me know if there are any other questions or concerns

[karelz]

Thanks for repro, we will take a look.

[karelz]

I see that your app targets netcoreapp20, can you please try it against .NET Core 2.1 to see if it still fails there? Thanks!

[spati2]

encountering the same exception. Do you want me to update the repro with TargetFramework netcoreapp21 change?

[karelz]

Sure, that won't hurt. Thanks for confirmation!

[spati2]

Hi @karelz. I hadnt correctly updated my repro. I installed the dotnet sdk version 2.1.300 and updated the framework to netcoreapp2.1 and my put request gets successfully processed. Closing this issue.. :)

[karelz]

OK, thanks for verifying!

[AlirezaSalimi1]

Hi, I am using asp net core 2.1.3 and when i want to postAsync await client.PostAsync(urlToCall.ToString(), streamContent); when i am in windows there is no problem but when my project is running on linux docker container i face error :

System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.InvalidOperationException: The stream was already consumed. It cannot be read again. at System.Net.Http.StreamContent.PrepareContent() at System.Net.Http.StreamContent.SerializeToStreamAsync(Stream stream, TransportContext context) at System.Net.Http.HttpContent.SerializeToStreamAsync(Stream stream, TransportContext context, CancellationToken cancellationToken) at System.Net.Http.HttpContent.CopyToAsync(Stream stream, TransportContext context, CancellationToken cancellationToken) at System.Net.Http.HttpConnection.SendRequestContentAsync(HttpRequestMessage request, HttpContentWriteStream stream, CancellationToken cancellationToken) at System.Net.Http.HttpConnection.SendAsyncCore(HttpRequestMessage request, CancellationToken cancellationToken)

[QHQuach]

We have a ASP .NET Core 1.1 project with .NET Framework 4.5.2 as the target. Upon upgrading it to .NET Framework 4.6.2 (in preparing for consuming a .NET Framework 4.6.2 nuget), we run into this exception. From the discussion, does it mean we also have to upgrade to .NET Core 2.1 even though we are on 1.1?

[davidsh]

From the discussion, does it mean we also have to upgrade to .NET Core 2.1 even though we are on 1.1?

If your issue is the same, then yes, you need to upgrade to .NET Core 2.1 (or .NET Core 2.2).

If this doesn't solve your problem, please open a new issue.

[ScottMathieson]

I believe this is actually still an issue when dealing with non-Seekable stream content. Seems to me that in the event of an authorization negotiation, there is a second attempt to read the stream content regardless if the underlying stream is seekable or not meaning this error happens. Not sure how this could be solved other than by using an OPTIONS request before attempting to POST to ensure the Negotiation phase happens before reading the content of the stream.

Not sure why this doesn't happen in .NET Framework

[stephentoub]

Not sure why this doesn't happen in .NET Framework

Potentially it's Expect: 100-continue kicking in? Using Expect: 100-continue defaults to true on .NET Framework, and false on .NET Core. It's possible the timing is working out such that on .NET Framework the sending of the request content is delayed sufficiently that it doesn't experience the issue. You could try disabling it on .NET Framework and see if the problem reproduces, and conversely try enabling it in .NET Core and see if it (non-deterministically) goes away.

[ScottMathieson]

Setting it doesn't have any effect so I think it must be something else. I can see in Fiddler the 401 response and the negotiate follow-up when running in .NET Framework. In dotnet core I simply see the 401 response and then the failure in StreamContent.PrepareContent() which looking at the source code can only happen when SerializeToStreamAsync gets called twice for the same stream. From the looks of things it doesn't seem likely that this would be a supported scenario with a non-Seekable stream (I'm using CryptoStream to upload an encrypted file). Looks like the only option is to read the stream into a memory stream or write it out to an intermediate file before streaming, neither of which are ideal,

As to why it works in .NET Framework, my only thought is that the body content is read once into memory and so never attempts to re-access the stream content in the event of a 401 Negotiate response.

Interestingly, when looking at the initial request in .NET Framework the content-length has been set so it must have read it into memory

[danielsh555]

Hi, We have a project with .NET Framework 4.7.2 as the target. We've received the same issue:

Error message = 'The stream was already consumed. It cannot be read again.'.,"System.InvalidOperationException Void PrepareContent() System.InvalidOperationException: The stream was already consumed. It cannot be read again.
   at System.Net.Http.StreamContent.PrepareContent()
   at System.Net.Http.StreamContent.SerializeToStreamAsync(Stream stream, TransportContext context)
   at System.Net.Http.HttpContent.LoadIntoBufferAsync(Int64 maxBufferSize)
   at System.Net.Http.HttpClient.StartContentBuffering(HttpRequestMessage request, CancellationTokenSource cancellationTokenSource, TaskCompletionSource`1 tcs, HttpResponseMessage response)
   at System.Net.Http.HttpClient.<>c__DisplayClass55_0.<SendAsync>b__0(Task`1 task)

So it seems that it doesn't work with .NET Framework too. Could you please help?

[ScottMathieson]

@danielsh555 - if you are having the same issue I was when using Windows Authentication and hitting issues with the 401 challenge response then you could try issuing a GET request to a known authenticated endpoint on the same server before making the POST request. That way the authentication challenge happens before the POST and allows the POST to complete.

It is a bit hacky but it is a possible workaround if you don't want to have to read the stream into a MemoryStream before posting.