JamesNK / Newtonsoft.Json

Json.NET is a popular high-performance JSON framework for .NET
https://www.newtonsoft.com/json
MIT License
10.71k stars 3.24k forks source link

Added MaxDepth property to JsonLoadSettings to override the maxDepth … #2904

Open tradercentric opened 11 months ago

tradercentric commented 11 months ago

…of 64 levels limit in JsonReader when the reader is called by JObject.Parse, or JArray.Parse.

tradercentric commented 11 months ago

Hi, this is a fix for Issue https://github.com/JamesNK/Newtonsoft.Json/issues/2900

tradercentric commented 11 months ago

I really need the issue fixed soon because I do have an Avro schema from the source team who cannot reduce the depth level the document from a vendor. I am open to alternative if my fix is not good enough. Please help.

sungam3r commented 11 months ago

@tradercentric if you in a hurry I really advise you to fork and compile your version. You may wait for applying/releasing for ages.

JamesNK commented 11 months ago

I don't think this feature is valuable. There is already a JObject.Load(JsonReader reader, JsonLoadSettings? settings) method that lets you customize the reader.

Having a setting on JsonLoadSettings for max depth creates confusion about which is used. And then the question comes up whether all the other settings on JsonReader should be added to JsonLoadSettings.

tradercentric commented 11 months ago

Thanks for the response. I am an user of the Apache Avro project which is using Newtonsoft's library, I will fork the Apache Avro project and see if I can instantiate JsonReader first before (to customize) before using JObject.Parse and JArray.Parse. That is to resolve the issue in https://github.com/apache/avro/blob/41b3c08ca5da192786c2b08546e691b3126e1856/lang/csharp/src/apache/main/Schema/Schema.cs#L250.

tradercentric commented 11 months ago

JsonReader.Load seems to count differently the depth levels of Avro schema, the following Avro schema has depth level 4. However in JsonReader's Push(JsonContainerType value), when I put a break point on the line "if (_maxDepth != null && Depth + 1 > _maxDepth && !_hasExceededMaxDepth)", it counted 11.

Here are the observation of over-counting depth level in Newtonsoft's JsonReader: Avro Schema Depth, JsonReader Depth Level Count 4, 11 16, 44 32, 92 64, 188 So, roughly speaking, the depth level count is about 2.75 times of Avro schema depth.

Here is the sample Avro schema for the test:

var json = @"{ ""type"": ""record"", ""name"": ""Level1"", ""fields"": [ { ""name"": ""field1"", ""type"": ""string"" }, { ""name"": ""field2"", ""type"": ""int"" }, { ""name"": ""level2"", ""type"": { ""type"": ""record"", ""name"": ""Level2"", ""fields"": [ { ""name"": ""field3"", ""type"": ""boolean"" }, { ""name"": ""field4"", ""type"": ""double"" }, { ""name"": ""level3"", ""type"": { ""type"": ""record"", ""name"": ""Level3"", ""fields"": [ { ""name"": ""field5"", ""type"": ""string"" }, { ""name"": ""field6"", ""type"": ""int"" }, { ""name"": ""level4"", ""type"": { ""type"": ""record"", ""name"": ""Level4"", ""fields"": [ { ""name"": ""field7"", ""type"": ""boolean"" }, { ""name"": ""field8"", ""type"": ""double"" } ] } } ] } } ] } } ] }";

I also tested a 32 depth level Avro schema, and the Push method internal depth level count is 92. Here is the 32 level depth Avro schema:

{ ""type"": ""record"", ""name"": ""Level1"", ""fields"": [ { ""name"": ""level2"", ""type"": { ""type"": ""record"", ""name"": ""Level2"", ""fields"": [ { ""name"": ""level3"", ""type"": { ""type"": ""record"", ""name"": ""Level3"", ""fields"": [ { ""name"": ""level4"", ""type"": { ""type"": ""record"", ""name"": ""Level4"", ""fields"": [ { ""name"": ""level5"", ""type"": { ""type"": ""record"", ""name"": ""Level5"", ""fields"": [ { ""name"": ""level6"", ""type"": { ""type"": ""record"", ""name"": ""Level6"", ""fields"": [ { ""name"": ""level7"", ""type"": { ""type"": ""record"", ""name"": ""Level7"", ""fields"": [ { ""name"": ""level8"", ""type"": { ""type"": ""record"", ""name"": ""Level8"", ""fields"": [ { ""name"": ""level9"", ""type"": { ""type"": ""record"", ""name"": ""Level9"", ""fields"": [ { ""name"": ""level10"", ""type"": { ""type"": ""record"", ""name"": ""Level10"", ""fields"": [ { ""name"": ""level11"", ""type"": { ""type"": ""record"", ""name"": ""Level11"", ""fields"": [ { ""name"": ""level12"", ""type"": { ""type"": ""record"", ""name"": ""Level12"", ""fields"": [ { ""name"": ""level13"", ""type"": { ""type"": ""record"", ""name"": ""Level13"", ""fields"": [ { ""name"": ""level14"", ""type"": { ""type"": ""record"", ""name"": ""Level14"", ""fields"": [ { ""name"": ""level15"", ""type"": { ""type"": ""record"", ""name"": ""Level15"", ""fields"": [ { ""name"": ""level16"", ""type"": { ""type"": ""record"", ""name"": ""Level16"", ""fields"": [ { ""name"": ""level17"", ""type"": { ""type"": ""record"", ""name"": ""Level17"", ""fields"": [ { ""name"": ""level18"", ""type"": { ""type"": ""record"", ""name"": ""Level18"", ""fields"": [ { ""name"": ""level19"", ""type"": { ""type"": ""record"", ""name"": ""Level19"", ""fields"": [ { ""name"": ""level20"", ""type"": { ""type"": ""record"", ""name"": ""Level20"", ""fields"": [ { ""name"": ""level21"", ""type"": { ""type"": ""record"", ""name"": ""Level21"", ""fields"": [ { ""name"": ""level22"", ""type"": { ""type"": ""record"", ""name"": ""Level22"", ""fields"": [ { ""name"": ""level23"", ""type"": { ""type"": ""record"", ""name"": ""Level23"", ""fields"": [ { ""name"": ""level24"", ""type"": { ""type"": ""record"", ""name"": ""Level24"", ""fields"": [ { ""name"": ""level25"", ""type"": { ""type"": ""record"", ""name"": ""Level25"", ""fields"": [ { ""name"": ""level26"", ""type"": { ""type"": ""record"", ""name"": ""Level26"", ""fields"": [ { ""name"": ""level27"", ""type"": { ""type"": ""record"", ""name"": ""Level27"", ""fields"": [ { ""name"": ""level28"", ""type"": { ""type"": ""record"", ""name"": ""Level28"", ""fields"": [ { ""name"": ""level29"", ""type"": { ""type"": ""record"", ""name"": ""Level29"", ""fields"": [ { ""name"": ""level30"", ""type"": { ""type"": ""record"", ""name"": ""Level30"", ""fields"": [ { ""name"": ""level31"", ""type"": { ""type"": ""record"", ""name"": ""Level31"", ""fields"": [ { ""name"": ""level32"", ""type"": ""string"" } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] } } ] }