JamesNK / Newtonsoft.Json

Json.NET is a popular high-performance JSON framework for .NET
https://www.newtonsoft.com/json
MIT License
10.77k stars 3.25k forks source link

JsonPath filters on multi-nav properties, but fails on single object #2386

Closed AKrasheninnikov closed 4 years ago

AKrasheninnikov commented 4 years ago

I'm struggling to understand what's wrong with my queries that attempt to traverse a hierarchical JSON output of SAP SuccessFactors OData endpoint. JSON Path recursive descent and filtering seems to work inconsistently: it works on arrays (JArray), but fails on single objects (JObject).

Source/destination types

No specific types, actually, just fiddling with JSON Path JToken.SelectToken or SelectTokens methods output. The business case was to process SAP SuccessFactors REST OData endpoint query results.

Source/destination JSON

Simplified JSON may look like this.

{
    "d": {
        "results": [{
            "__metadata": {
                "type": "SFOData.PerPersonal"
            },
            "property": "value",
            "objectNav": {
                "__metadata": {
                "type": "SFOData.PerPerson"
                },
                "personIdExternal":"1234",
                "arrayNav": { 
                    "results": [{
                                "__metadata": {
                                    "type": "SFOData.PerPhone"
                                },
                                "phoneType": "21735",
                                "phoneNumber": "+7 (000) 000-42-34"
                            }, {
                                "__metadata": {
                                    "type": "SFOData.PerPhone"
                                },
                                "phoneType": "10876", 
                                "phoneNumber": "775715"
                            }]
                    }
                }
            }]      
    }
}

Expected behavior

I expect the SelectToken/SelectTokens with recursive descent operator to work consistently regardless of the presence of array or lack thereof.

Actual behavior

Successful query JSON Path "$..*[?(@.__metadata.type=='SFOData.PerPhone')].phoneNumber" yields "775715"
No results for JSON Path "$..*[?(@.__metadata.type=='SFOData.PerPerson')].personIdExternal"

Steps to reproduce

        [Test]
        public async Task SfODataDemo()
        {
            JToken sfodataJson;
            string[] successfulQueries = new [] {
              "$..results[?(@.__metadata.type=='SFOData.PerPhone')].phoneNumber",
              "$..*[?(@.__metadata.type=='SFOData.PerPhone')].phoneType",
              "$..*[?(@.__metadata.type=='SFOData.PerPhone')].phoneNumber"
            }, failingQueries = new [] {
              "$..*[?(@.__metadata.type=='SFOData.PerPerson')].personIdExternal",
            };

            // read asynchronously from a file
            using (TextReader textReader = new StreamReader(new FileStream(ResolvePath(@"sfodata.json"), FileMode.Open, FileAccess.Read, FileShare.Read, 4096, true)))
            {
                sfodataJson = await JToken.LoadAsync(new JsonTextReader(textReader));
            }

            foreach (string jsonPath in successfulQueries){
              JToken record = sfodataJson.SelectToken(jsonPath);
            }

            foreach (string jsonPath in failingQueries){
              JToken record = sfodataJson.SelectToken(jsonPath);
            }

        }
JamesNK commented 4 years ago

_metadata has one underscore.

AKrasheninnikov commented 4 years ago

Okay, James, thank you, you're right, I made a typo, but it doesn't change the behavior.

AKrasheninnikov commented 4 years ago

Corrected the underscores

AKrasheninnikov commented 4 years ago

The simplified JSON was typed in manually, but the query was constructed for a real output with no such typos. I apologise for a little confusion, but I hope we can improve the great tool and make it even better.

AKrasheninnikov commented 4 years ago

Although at first I've messed the braces, trying to convey the idea in a simpler form, I apologise and beg you to take the corrected version into consideration.

{
    "d": {
        "results": [{
            "__metadata": {
                "type": "SFOData.PerPersonal"
            },
            "property": "value",
            "objectNav": {
                "__metadata": {
                "type": "SFOData.PerPerson"
                },
                "personIdExternal":"1234",
                "arrayNav": { 
                    "results": [{
                                "__metadata": {
                                    "type": "SFOData.PerPhone"
                                },
                                "phoneType": "21735",
                                "phoneNumber": "+7 (000) 000-42-34"
                            }, {
                                "__metadata": {
                                    "type": "SFOData.PerPhone"
                                },
                                "phoneType": "10876", 
                                "phoneNumber": "775715"
                            }]
                    }
                }
            }]      
    }
}
AKrasheninnikov commented 4 years ago

So I took the sources and by debugging, found that * is to blame in some cases.

Having processed the same JSON with JsonPath expressions differing only in the asterisk before the brackets, the one without asterisk works as expected, while the one with an asterisk returns an empty set.

JsonPath query should succeed:  "$..[?(@.__metadata.type=='SFOData.FODepartment')].name_ru_RU"
[SUCCESS] at 109:43 found {"__metadata": {
  "uri": "https://demo.sapsf.com/odata/v2/FODepartment(externalCode='DEP002051',startDate=datetime'2019-11-01T00:00:00')",
  "type": "SFOData.FODepartment"
}}
...
[SUCCESS] at 409:43 found {"__metadata": {
  "uri": "https://demo.sapsf.com/odata/v2/FODepartment(externalCode='DEP001148',startDate=datetime'2019-11-01T00:00:00')",
  "type": "SFOData.FODepartment"
}}
[SUCCESS] [RESULT] {Отдел блаблабла}

JsonPath query should succeed:  "$..*[?(@.__metadata.type=='SFOData.FODepartment')].name_ru_RU"
JsonPath query FAILED "$..*[?(@.__metadata.type=='SFOData.FODepartment')].name_ru_RU" returned an empty set

image