restsharp / RestSharp

Simple REST and HTTP API Client for .NET
https://restsharp.dev
Apache License 2.0
9.58k stars 2.34k forks source link

Deserialization fails when Content-Type is null #789

Closed Tylerflick closed 5 years ago

Tylerflick commented 8 years ago

I recently found a bug where a DELETE response object causes a null reference exception when the response meets the following:

This most likely affects all request types.

Looking through the source the issue appears to be line 624 in RestSharp/RestClient.cs where raw.ContentType is being accessed: if (response.ErrorException == null) { IDeserializer handler = this.GetHandler(raw.ContentType);

There should either be a null check here, or the IRestResponse object raw should be passed into GetHandler() and the null check should be done there.

hallem commented 8 years ago

i agree the code should fail more gracefully but content-type is a required field for restsharp. i'll see what i can do to make this fail better.

uxsoft commented 8 years ago

I was not able to reproduce this issue. How did you get to a state where your response had body and content type null? If i point RestSharp to a server returning an empty response it is handled by RestSharp correctly.

Moreover what object was actually null? if raw.ContentType == null Deserialize doesn't throw and GetHandler checks for null contentType. If raw is null then that is an expected outcome. No codepaths in RestSharp generate a null response.

Unless you can provide a code sample to reproduce this issue I consider it resolved.

Tylerflick commented 8 years ago

I ended up using a different version of the library which did not propagate the error. I'm fine with closing the issue out.

uxsoft commented 8 years ago

And could you please test it with the version now in master? Maybe the null checks were added after you opened this. If the issue still appears could you please provide a code sample?

If i can reproduce it I can fix it for everybody:)

Tylerflick commented 8 years ago

Maybe the null checks were added after you opened this

They may very well have.

It might be a bit before I can get to it as it was for a project at work which I'm no longer working on. Sorry for not being able to check on this at the moment.

Qwin commented 7 years ago

I got the same issue, and I know why its happening. The following: await Client.ExecuteTaskAsync<T> (request, token);

Say I put for T is Object. However the server that I am calling won't return any content. Thus the ContentType would be null and the GetHandler would throw a ArgumentNullException. If I had done: await Client.ExecuteTaskAsync(request, token);

It wouldn't have thrown an error. Mainly because it doesn't expect anything to deserialize. What I would like to see is that when the contentType is null dont throw an error and just handle it the same way as if there wasn't any content returned.

Kevin-Skeepers commented 6 years ago

Hello, I had the same issue as @Qwin I think. I'm using Polly to wrap RestSharp calls.

And I had some calls that were retrying for no reason: it's because RestSharp itself was throwing an ArgumentNullException, I investigated real quick and managed to grab the content-type null error that RestSharp was throwing. My calls were being retried because RestSharp was going crazy.

Here is my setup: The client POST or DELETE to my server using await Client.ExecuteTaskAsync<T> (request); Sometimes the server replies BadRequest.

When ExecuteTaskAsync is typed then I got the RestSharp content-type error. There is no content-type since the method is POST or DELETE. When ExecuteTaskAsync is not typed then there is no error.

Thanks @Qwin to have mentioned this error, you made me save a solid day of investigating, I made separate calls to ExecuteTaskAsync without type when it's a POST/DELETE/PUT and now my code works fine.

alexeyzimarev commented 6 years ago

Well, as far I can see in the code, we assign deserialiser based on content type. If it is null - we have no idea what to do. But I will have to look once more.

Kevin-Skeepers commented 6 years ago

Yeah, makes sense. Maybe it should be just extra documentation? Or, when the method is not GET then ignore the serialize, type, etc. ?

jdextraze commented 6 years ago

I also experience that issue on mono. After looking into the implementation, it seems mono return a null content type from the HttpWebResponse when the header is not present. Other implementation return string.Empty.

Mono source | .Net Core source | .Net Framework source decompile:

[__DynamicallyInvokable]
public override string ContentType
{
  [__DynamicallyInvokable] get
  {
    this.CheckDisposed();
    return this.m_HttpResponseHeaders.ContentType ?? string.Empty;
  }
}
amolsarmalkar commented 5 years ago

Hi all, Our solution consists of 3 projects .

One is shared project whose dll is referenced in iOS project as well as Android. In shared project we use Rest client with RestSharp library version 105.2.3. However the api fails if token is expired. We used ExecuteTaskAsync function of Restclient and it is GET request. ExecuteTaskAsync generates web exception.

This code statement that generates exception var response = await client.ExecuteTaskAsync(request);

The web exception it generates is : Error getting response stream (ReadAsync): ReceiveFailure Value cannot be null. Parameter name: src

Restclient works properly if used with Visual studio community edition and also works fine for Previous version of Profession edition . But it fails with Visual studio Professional edition latest version version 7.6.9 (Xamarin.iOS 12.1.0.15) Other apis which do not need authentication work fine.

I tried latest RestSharp library version 106.5.4 but it is causing same issue.

One more observation : We have web services running on UAT server as well as Live server. But on UAT our code works just fine. There is Difference in response headers of UAT and LIVE. UAT response headers contain 2 additional headers

  1. connection →keep-alive 2.transfer-encoding →chunked

I have yet to explore the relation of these headers with the issue we are facing.

Please suggest me the best approach to resolve this error.

alexeyzimarev commented 5 years ago

We need, as described in the contribution guidelines:

amolsarmalkar commented 5 years ago

Thanks alexeyzimarev for showing interest in this issue

  1. Web exception details {System.Net.WebException: Value cannot be null. Parameter name: src ---> System.ArgumentNullException: Value cannot be null. Parameter name: src at System.Buffer.BlockCopy (System.Array src, System.Int32 srcOffset, System.Array dst, System.Int32 dstOffset, System.Int32 count) [0x00003] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.10/src/Xamarin.iOS/mcs/class/corlib/ReferenceSources/Buffer.cs:39 at System.Net.WebResponseStream+d49.MoveNext () [0x00082] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.10/src/Xamarin.iOS/mcs/class/System/System.Net/WebResponseStream.cs:203 --- End of inner exception stack trace --- at System.Net.HttpWebRequest+d2411[T].MoveNext () [0x000b5] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.10/src/Xamarin.iOS/mcs/class/System/System.Net/HttpWebRequest.cs:951 --- End of stack trace from previous location where exception was thrown --- at System.Net.WebConnectionStream.Read (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00070] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.10/src/Xamarin.iOS/mcs/class/System/System.Net/WebConnectionStream.cs:136 at RestSharp.Extensions.MiscExtensions.ReadAsBytes (System.IO.Stream input) [0x0001c] in <bdc831d0d5cd47b69e345202ae683d3a>:0 at RestSharp.Http.ProcessResponseStream (System.IO.Stream webResponseStream, RestSharp.HttpResponse response) [0x00008] in <bdc831d0d5cd47b69e345202ae683d3a>:0 at RestSharp.Http.ExtractResponseData (RestSharp.HttpResponse response, System.Net.HttpWebResponse webResponse) [0x0003a] in <bdc831d0d5cd47b69e345202ae683d3a>:0 at RestSharp.Http+<>c__DisplayClass15.<ResponseCallback>b__13 (System.Net.HttpWebResponse webResponse) [0x00000] in <bdc831d0d5cd47b69e345202ae683d3a>:0 at RestSharp.Http.GetRawResponseAsync (System.IAsyncResult result, System.Action1[T] callback) [0x00046] in :0 at RestSharp.Http.ResponseCallback (System.IAsyncResult result, System.Action`1[T] callback) [0x00056] in :0 }

########

  1. System.WebException stack trace in RestSharp response is .

    at System.Net.HttpWebRequest+d241`1[T].MoveNext () [0x000b5] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.10/src/Xamarin.iOS/mcs/class/System/System.Net/HttpWebRequest.cs:951 --- End of stack trace from previous location where exception was thrown --- at System.Net.WebConnectionStream.Read (System.Byte[] buffer, System.Int32 offset, System.Int32 count) [0x00070] in /Library/Frameworks/Xamarin.iOS.framework/Versions/12.2.1.10/src/Xamarin.iOS/mcs/class/System/System.Net/WebConnectionStream.cs:136 at RestSharp.Extensions.MiscExtensions.ReadAsBytes (System.IO.Stream input) [0x0001c] in :0 at RestSharp.Http.ProcessResponseStream (System.IO.Stream webResponseStream, RestSharp.HttpResponse response) [0x00008] in :0 at RestSharp.Http.ExtractResponseData (RestSharp.HttpResponse response, System.Net.HttpWebResponse webResponse) [0x0003a] in :0 at RestSharp.Http+<>cDisplayClass15.b__13 (System.Net.HttpWebResponse webResponse) [0x00000] in :0 at RestSharp.Http.GetRawResponseAsync (System.IAsyncResult result, System.Action1[T] callback) [0x00046] in <bdc831d0d5cd47b69e345202ae683d3a>:0 at RestSharp.Http.ResponseCallback (System.IAsyncResult result, System.Action1[T] callback) [0x00056] in :0

3.I am attaching 2 screenshots of response. Please check those screenshots . Please let me know whether these screenshots reveal Raw response and request data .Otherwise I will send request url and parameters by asking permission from my official heads.

RAW response I can get from Postman client ( Chrome browser addon). Postman client is able to read response perfectly with proper status code.

4.Code with context -: Please let me know in details the information you need. The code statement that generates exception is var response = await client.ExecuteTaskAsync(request); Also tried the variation of above api i.e., var response = await client.ExecuteTaskAsync(request);

In Parameter T we pass the response class in which we deserialise response data.

screen shot 2018-12-18 at 8 41 22 pm screen shot 2018-12-18 at 8 45 00 pm

alexeyzimarev commented 5 years ago

Ok, it is fixed here https://github.com/restsharp/RestSharp/commit/6ede4d2ecb9d52503b9075d82c6f7128d1f20d15 but you need to have a handler with the content type *, which is currently XML by default. If you need it to be JSON, use client.UseJson(), it will completely remove XML handling from the client.

amolsarmalkar commented 5 years ago

Ok, it is fixed here 6ede4d2 but you need to have a handler with the content type *, which is currently XML by default. If you need it to be JSON, use client.UseJson(), it will completely remove XML handling from the client.

alexeyzimarev , in what version of Restsharp nuget package this change will get included? And when it will become available?

alexeyzimarev commented 5 years ago

I plan to make a release this week, v106.7, with many previously open issues addressed in the last release with SimpleJson. The next release after that would be a major version.

amolsarmalkar commented 5 years ago

Hi alexeyzimarev,

Are the changes commited in 6ede4d2 available with version 106.6.7 ? Or we need to wait for version release of 106.7 .

beeradmoore commented 5 years ago
  1. connection →keep-alive 2.transfer-encoding →chunked

I have yet to explore the relation of these headers with the issue we are facing.

@amolsarmalkar , we had the same issue with running ASP.NET locally or via AWS Lambda. Difference we had was transfer-encoding as chunked was only happening on Lambda. We had to do this funky workaround to get transfer-type chunked out of there. Having the content-length header set (as a result of removing chunked transfer type) fixed the issue for us. I am curious to see if 106.7 causes the same issue on our end here when not running the de-chunker.

alexeyzimarev commented 5 years ago

My change will be available in 106.7

amitlodha commented 5 years ago

any update on above release 106.7

carvalho-oak commented 5 years ago

Please provide an update on this fix. We really need the 106.7 out... thanks.

alexeyzimarev commented 5 years ago

The latest version has been released almost a week go

amitlodha commented 5 years ago

@alexeyzimarev I can see the latest version as 106.6.9 and not 106.7 so above changes are available in 106.6.9?

https://www.nuget.org/packages/RestSharp/106.6.9

alexeyzimarev commented 5 years ago

@amitlodha All changes are in the latest available version.

amolsarmalkar commented 5 years ago

I tried with latest version 106.6.9 and the issue seen is same as observed with previous versions of RestSharp. We have used OAuth2 and we pass Access token in GET request. As long as the access token is alive, we get proper response. Once the access token expires, the response is received with blank content type , status code 0, content length 0 .Why this scenario fails?

Code we used is as follows: var request = new RestRequest("/Endpoint Name/" + 123456765432, Method.GET);

           request.AddParameter("accessToken", "Access token cached");

            RestClient client = new RestClient("Server url");
            client.UseJson();

            request.RequestFormat = DataFormat.Json;
            request.Timeout = 20000;

           // Get the response
            var response = await client.ExecuteTaskAsync(request);

screen shot 2019-03-05 at 1 02 12 pm

yongyong-john commented 5 years ago

@hallem @alexeyzimarev, Would you mind, can you tell me when the v106.7 is coming? I have same error like @amolsarmalkar 's comment.

carvalho-oak commented 5 years ago

My error is now fixed by updating to the latest mono/xamarin version. The issue was not in the restsharp library but in the mono framework.

On May 16, 2019, at 6:23 PM, HonorFace-Shin notifications@github.com wrote:

Would you mind, can you tell me when the v106.7 is coming? I have same error like another comment.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_restsharp_RestSharp_issues_789-3Femail-5Fsource-3Dnotifications-26email-5Ftoken-3DAAIYC4YJE47KQ6NF4CIOPOLPVYCHTA5CNFSM4BWIS3X2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVTO2HA-23issuecomment-2D493284636&d=DwMCaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=cq7eNSIKV9hvlkGIuwx8gg&m=frfcZ62ZhPVUTZjT__QrVQG3QqFjV2plg23O4upSRt0&s=CGeT_MYxZ1CvSk_prv0jINALUp1JfDV-BogYTQePTDs&e=, or mute the thread https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AAIYC43JNRR3RRHWHK4BL33PVYCHTANCNFSM4BWIS3XQ&d=DwMCaQ&c=r2dcLCtU9q6n0vrtnDw9vg&r=cq7eNSIKV9hvlkGIuwx8gg&m=frfcZ62ZhPVUTZjT__QrVQG3QqFjV2plg23O4upSRt0&s=6aK0FE4N7U9roR-LTQj_aREoLZ3mkrl7TE1tFH8UQRA&e=.

yongyong-john commented 5 years ago

@carvalho-oak Could you tell me what kind of version do you used? Please tell me mono, xamarin forms(+ xamarin mac, ios, andoird), visual studio and restsharp version.

shirshGit commented 3 years ago

I am having similar issue where from one machine we are able to access the BearerTokenRespose properly(restclient.execute(bearerTokenRequest)) and it returns access token in the 'Content' of the Response. But In a VM when running same code the 'Content' is empty.

BearerTokenContent bearerToken = null; var bearerTokenRequest = new RestRequest("token"); bearerTokenRequest.Method = Method.POST; bearerTokenRequest.AddParameter("Content-Type", "application/x-www-form-urlencoded", ParameterType.HttpHeader); bearerTokenRequest.AddParameter("Accept", "/", ParameterType.HttpHeader); bearerTokenRequest.AddParameter("grant_type", "client_credentials"); bearerTokenRequest.AddParameter("client_id", client_id); bearerTokenRequest.AddParameter("client_secret", client_secret);

IRestResponse BearerTokenResponse = restClient.Execute(bearerTokenRequest);

Not able to resolve it.

Note: Upgraded to the latest version(106.11.7.0) of RestSharp but does not resolve the issue.

Can someone help to resolve it.