search

JamesNK / Newtonsoft.Json

Json.NET is a popular high-performance JSON framework for .NET
https://www.newtonsoft.com/json
MIT License
10.73k stars 3.25k forks source link

Deserializing to proxy type throws "Member already exists" error in WebAssembly #2836

Open smfields opened 1 year ago

smfields commented 1 year ago

Source/destination types

public interface IMyInterface
{
    public string MyProperty { get; set; }
}

Source/destination JSON

{"MyProperty": "Hello"}

CustomCreationConverter

public class ProxyConverter<T> : CustomCreationConverter<T>
{
    private static readonly ProxyGenerator Generator = new ();

    public override T Create(Type objectType)
    {
        var proxy = Generator.CreateInterfaceProxyWithoutTarget(typeof(T), new ProxyInterceptor<T>());
        return (T) proxy;
    }
}

Expected behavior

The proxy converter should create a new proxy type that is populated during deserialization.

Actual behavior

Deserialization works properly in a regular .NET application, such as a console app, but fails when running in WebAssembly. In WebAssembly, the following error is produced:

Unhandled exception rendering component: A member with the name '' already exists on 'Castle.Proxies.IMyInterfaceProxy'. Use the JsonPropertyAttribute to specify another name.
Newtonsoft.Json.JsonSerializationException: A member with the name '' already exists on 'Castle.Proxies.IMyInterfaceProxy'. Use the JsonPropertyAttribute to specify another name.
   at Newtonsoft.Json.Serialization.JsonPropertyCollection.AddProperty(JsonProperty property)
   at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateConstructorParameters(ConstructorInfo constructor, JsonPropertyCollection memberProperties)
   at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateObjectContract(Type objectType)
   at Newtonsoft.Json.Serialization.DefaultContractResolver.CreateContract(Type objectType)
   at System.Collections.Concurrent.ConcurrentDictionary`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].GetOrAdd(Type key, Func`2 valueFactory)
   at Newtonsoft.Json.Utilities.ThreadSafeStore`2[[System.Type, System.Private.CoreLib, Version=7.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Newtonsoft.Json.Serialization.JsonContract, Newtonsoft.Json, Version=13.0.0.0, Culture=neutral, PublicKeyToken=30ad4fe6b2a6aeed]].Get(Type key)
   at Newtonsoft.Json.Serialization.DefaultContractResolver.ResolveContract(Type type)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Populate(JsonReader reader, Object target)
   at Newtonsoft.Json.Serialization.JsonSerializerProxy.PopulateInternal(JsonReader reader, Object target)
   at Newtonsoft.Json.JsonSerializer.Populate(JsonReader reader, Object target)
   at Newtonsoft.Json.Converters.CustomCreationConverter`1[[Shared.IMyInterface, Shared, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]].ReadJson(JsonReader reader, Type objectType, Object existingValue, JsonSerializer serializer)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.DeserializeConvertable(JsonConverter converter, JsonReader reader, Type objectType, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonConverter[] converters)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[IMyInterface](String value, JsonConverter[] converters)
   at Shared.InstanceCreator.CreateInstance() in C:\repos\ConsoleApp\Shared\InstanceCreator.cs:line 9
   at WebAssemblyApp.Pages.Index.OnInitialized() in C:\repos\ConsoleApp\WebAssemblyApp\Pages\Index.razor:line 12
   at Microsoft.AspNetCore.Components.ComponentBase.RunInitAndSetParametersAsync()

Steps to reproduce

Reproduction Repo: https://github.com/smfields/NewtonsoftIssueRepro

  1. Run the ConsoleApp project to see the expected behaviour.
  2. Run the WebAssemblyApp project to see the error.
elgonzo commented 1 year ago

What you see could be the effect of trimming applied to the Blazor webassembly project (note the empty member name in the exception message, which could be an indication of this).

Therefore, first verify if trimming is the actual cause of your problem by disabling trimming for your Blazor webassembly project and check whether the problem still occurs without trimming.

If you have identified trimming to be the cause of the problem, evaluate whether you can switch to System.Text.Json, a json serializer using code generators. Because Newtonsoft.Json/Json.NET relies heavily on reflection, and reflection-based code often does not play very well with trimming without special care.

Quote from https://learn.microsoft.com/en-gb/dotnet/core/deploying/trimming/prepare-libraries-for-trimming:

Sometimes the existing design of an API will render it mostly trim-incompatible, and you may need to find other ways to accomplish what it is doing. A common example is reflection-based serializers. In these cases, consider adopting other technology like source generators to produce code that is more easily statically analyzed.

(Emphasis mine)

If you prefer to use Newtonsoft.Json/Json.NET while still trimming your project, you will have to configure your Blazor web assembly in a manner so that the trimmer does not trim away the types or the respective type members of any possible (data) objects and interfaces that potentially participating deserialization - such as properties, fields and especially constructors that will only be accessed by the Json.NET (de)serializer through reflection. For documentation/guidance, see here: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trimming-options

smfields commented 1 year ago

What you see could be the effect of trimming applied to the Blazor webassembly project (note the empty member name in the exception message, which could be an indication of this).

Therefore, first verify if trimming is the actual cause of your problem by disabling trimming for your Blazor webassembly project and check whether the problem still occurs without trimming.

If you have identified trimming to be the cause of the problem, evaluate whether you can switch to System.Text.Json, a json serializer using code generators. Because Newtonsoft.Json/Json.NET relies heavily on reflection, and reflection-based code often does not play very well with trimming without special care.

Quote from https://learn.microsoft.com/en-gb/dotnet/core/deploying/trimming/prepare-libraries-for-trimming:

Sometimes the existing design of an API will render it mostly trim-incompatible, and you may need to find other ways to accomplish what it is doing. A common example is reflection-based serializers. In these cases, consider adopting other technology like source generators to produce code that is more easily statically analyzed.

(Emphasis mine)

If you prefer to use Newtonsoft.Json/Json.NET while still trimming your project, you will have to configure your Blazor web assembly in a manner so that the trimmer does not trim away the types or the respective type members of any possible (data) objects and interfaces that potentially participating deserialization - such as properties, fields and especially constructors that will only be accessed by the Json.NET (de)serializer through reflection. For documentation/guidance, see here: https://learn.microsoft.com/en-us/dotnet/core/deploying/trimming/trimming-options

Thanks for your quick (and thorough) repsonse. Forgive me if I've misunderstood, I'm not very familiar with the idea of trimming.

I tried disabling trimming in the WebAssembly project by adding <PublishTrimmed>false</PublishTrimmed> and/or <TrimMode>partial</TrimMode> to the csproj. The error still reproduced, but I'm not entirely sure if this actually disables trimming or not.

I also tried explicitly turning on trimming for the ConsoleApp to see if I could get it to reproduce the error, but it did not. That being said, it looks like starting in .NET 7 all assemblies have trimming enabled by default. Again, forgive me if I've misunderstood.

I'd really prefer to stick with JSON.NET in this case, so are there any other potential causes for this error? It's unfortunately quite difficult to debug 3rd party libraries in WASM, so I can't easily dig into things myself.

elgonzo commented 1 year ago

Hmm, looking closer, you are uisng Castle DynamicProxy, which itself uses System.Reflection.Emit, apparently to create proxy types dynamically at runtime. (My suspicion regarding trimming was a false flag. My apologies.)

Considering this and your stacktrace that indicates Json.NET inspecting some ConstructorInfo for the dynamically generated proxy type that then leads it to detect a constructor parameter with no/empty name, i took a look at the respective method in Json.NET. (Side note: look at the code comment inside that method there, lol):

https://github.com/JamesNK/Newtonsoft.Json/blob/0a2e291c0d9c0c7675d445703e51750363a549ef/Src/Newtonsoft.Json/Serialization/DefaultContractResolver.cs#L687-L716

Note how the constructor parameter names are tested against null. But they are not tested against an empty string. I don't know if Castle can create constructor parameters for a proxy with a name that is not null but an empty string under certain circumstances, but it almost looks like that. And if that were to be the case, then Json.NET is not yet prepared for this (and therefore this being a bug/limitation).

Unfortunately, i am unable to set up and test your webassembly project (nor do i have much technical experience with wasm anyways), so i can't check for myself. If you want to, perhaps try reflecting over the type of the generated proxy instance when running as part of your webassembly project, and see/log if the parameter names of the constructor(s) of this type are either null, an empty string or some other name consisting only of some weird Unicode zero-width control/format characters. Not that this will help you fixing or working around the issue, but it could help confiriming what the actual constructor parameter names of the proxy object are when running in the webassembly...

smfields commented 1 year ago

I setup the test you recommended and as you predicated it looks like the constructor parameter names are being set to null in the ConsoleApp, but an empty string in the WebAssemblyApp.

elgonzo commented 1 year ago

(Sorry, i deleted my last comment. I misread your comment, falsely reading that the ctor parameter names are null in the WebAssemblyApp. Ooops...)

an empty string in the WebAssemblyApp.

Alright. That is then an issue with Json.NET library...

RagavanPV commented 5 months ago

Is there any workaround or a resolution for this issue. Have been facing this issue with a weird usecase, My API deserialization works with Debug configuration but throws error in Release configuration Newtonsoft.Json.JsonSerializationException: A member with the name '' already exists on 'System.Tuple2[System.String,System.String]'. Use the JsonPropertyAttribute to specify another name

BhanuGoyal-AQI commented 2 months ago

@RagavanPV Did you get any solution for it? I am also facing similar issue...