Exception during deserialization with TypeNameHandling = TypeNameHandling.All a json that was saved by browser

slavapvf commented 4 months ago

Source/destination types

public class Test1
{
    public Dictionary<string, decimal> Data { get; set; }
}

Source/destination JSON

{
    "$type": "Test.Core.Test1, Test.Core",
    "Data": {
        "0": 1,
        "1": 2,
        "2": 3,
        "$type": "System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib],[System.Decimal, System.Private.CoreLib]], System.Private.CoreLib"
    }
}

Actual behavior

Exception: "Could not convert string to decimal: System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib],[System.Decimal, System.Private.CoreLib]], System.Private.CoreLib. Path 'Data.$type', line 1, position 231." | string

| Path | "Data.$type" | string

at Newtonsoft.Json.JsonReader.ReadDecimalString(String s) at Newtonsoft.Json.JsonTextReader.FinishReadQuotedNumber(ReadType readType) at Newtonsoft.Json.JsonTextReader.ReadNumberValue(ReadType readType) at Newtonsoft.Json.JsonTextReader.ReadAsDecimal() at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateDictionary(IDictionary dictionary, JsonReader reader, JsonDictionaryContract contract, JsonProperty containerProperty, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue(JsonProperty property, JsonConverter propertyConverter, JsonContainerContract containerContract, JsonProperty containerProperty, JsonReader reader, Object target) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue) at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent) at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType) at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType) at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader) at PMOExpress.BL.Tests.MetadataExtensionsTests.Test1() in C:\Projects\Sources\Tests\Data.Tests\ExtensionsTests.cs:line 517

Steps to reproduce

Create a controller, that returns a json:

public ActionResult TestJson()
{
    var s = new Test1()
    {
        Data = new Dictionary<string, decimal>()
        {
            { "0", 1 },
            { "1", 2 },
            { "2", 3 },
        }
    };
    var str = JsonConvert.SerializeObject(s, new JsonSerializerSettings()
    {
        TypeNameHandling = TypeNameHandling.All
    });
    return Content(str, "application/json");
}

Navigate to controller in Chrome or IE and save result to a file. Notice that property"$type" is in the last position as browser sorts properties of json object.
Create test and paste context of the file to variable. I already did it for variable str


    [Fact]
    public void Test1()
    {
        var str = "{\"$type\":\"MyProject.Core.Test1, MyProject.Core\",\"Data\":{\"0\":1,\"1\":2,\"2\":3,\"$type\":\"System.Collections.Generic.Dictionary`2[[System.String, System.Private.CoreLib],[System.Decimal, System.Private.CoreLib]], System.Private.CoreLib\"}}";
        var reader = new StringReader(str);
        using var jsonreader = new JsonTextReader(reader);
        var demoData = JsonSerializer
            .Create(new JsonSerializerSettings() { TypeNameHandling = TypeNameHandling.All })
            .Deserialize<PMOExpress.Core.Test1>(jsonreader);
    }

Run test, Exception occurs. I suspect because of the position of $type property

elgonzo commented 4 months ago

I suspect because of the position of $type property

Correct. By default, Newtonsoft.Json detects metadata keywords such as $type only at the beginning of json objects (for performance and memory reasons).

In your situation, you need to explicitly instruct Newtonsoft.Json to read ahead in the json data to look for metadata keywords, at the cost of buffering json data during deserialization. This is done by setting the JsonSerializerSettings MetadataPropertyHandling property to ReadAhead. (https://www.newtonsoft.com/json/help/html/SerializationSettings.htm#MetadataPropertyHandling)

slavapvf commented 4 months ago

Thank you for solution. It is working now. Would be nice to have a hint on TypeNameHandling with recommendation for MetadataPropertyHandling because the exception can happens occasionally.

JamesNK / Newtonsoft.Json