Orange-OpenSource / hurl

Hurl, run and test HTTP requests with plain text.
https://hurl.dev
Apache License 2.0
13.17k stars 493 forks source link

Using jsonpath filter on object variable fails #3369

Open luke-pp opened 3 weeks ago

luke-pp commented 3 weeks ago

In my company's GraphQL APIs, we can get back some deeply nested json responses with a lot of data, and often we want to perform many different assertions against various parts of the response - ideally we'd like to organize these assertions by capturing one part of the response, and then performing assertions on each captured subsection separately. Here's a simplified example using the star wars API to show what I mean:

POST https://swapi-graphql.netlify.app/.netlify/functions/index
```graphql
query ExampleQuery($first: Int, $after: String) {
  allPlanets(first: $first, after: $after) {
    edges {
      cursor
      node {
        name 
        diameter
        population
        filmConnection {
          films {
            title
          }
        }
      }
    }
  }
}
variables {
    "first": 10
}

HTTP 200 [Captures] bespin: jsonpath "$.data.allPlanets.edges[?(@.node.name == 'Bespin')]" nth 0 tatooine: jsonpath "$.data.allPlanets.edges[0]"

[Asserts] variable "bespin" jsonpath "$.node.diameter" == 118000 variable "bespin" jsonpath "$.node.population" == 6000000

variable "tatooine" jsonpath "$.node.diameter" == 10465 variable "tatooine" jsonpath "$.node.population" == 200000

In this example, we want to do a couple of assertions about each planet, so we first capture the json objects for Bespin and Tatooine, and then perform assertions against these objects using the jsonpath filter. We could get the same behaviour by just using jsonpath assertions along the lines of

jsonpath "$.data.allPlanets.edges[?(@.node.name == 'Bespin')].node.diamter" nth 0 == 118000

but you can imagine it gets a bit more frustrating in a real workflow outside of the star wars API example.

### What is the current *bug* behavior?
I get errors in the console:
error: Filter error --> .\hurl-issue.hurl:31:19 POST https://swapi-graphql.netlify.app/.netlify/functions/index ... 31 variable "bespin" jsonpath "$.node.diameter" == 118000 ^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object
error: Filter error --> .\hurl-issue.hurl:32:19 POST https://swapi-graphql.netlify.app/.netlify/functions/index ... 32 variable "bespin" jsonpath "$.node.population" == 6000000 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object
error: Filter error --> .\hurl-issue.hurl:34:21 POST https://swapi-graphql.netlify.app/.netlify/functions/index ... 34 variable "tatooine" jsonpath "$.node.diameter" == 10465 ^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object
error: Filter error --> .\hurl-issue.hurl:35:21 POST https://swapi-graphql.netlify.app/.netlify/functions/index ... 35 variable "tatooine" jsonpath "$.node.population" == 200000 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ invalid filter input: object

### Steps to reproduce
Run the hurl file above

### What is the expected *correct* behavior?
Ideally, the jsonpath should work on the variable object in the same way it works on a full response body/string such that the assertions in the example file pass.

### Execution context
I don't think it's particularly relevant to this issue, but

- Hurl Version (`hurl --version`):
hurl 4.3.0 (Windows) libcurl/8.4.0-DEV Schannel zlib/1.3 nghttp2/1.58.0
Features (libcurl):  alt-svc AsynchDNS HSTS HTTP2 IPv6 Largefile libz NTLM SPNEGO SSL SSPI Unicode UnixSockets
Features (built-in): brotli

### Possible fixes
I believe the issue is here:
https://github.com/Orange-OpenSource/hurl/blob/9cfc5b90b0d96583319350a803fa453f8a167497/packages/hurl/src/runner/filter/jsonpath.rs#L33

The jsonpath filter runner currently only accepts string values, and assumes it needs to parse the string into a json value before evaluating the json path. In our example, the first jsonpath evaluates to fetch the full planet object and capture it to a variable as an object which works fine. But then when we try to use the jsonpath filter on the object to perform assertions against specific items in it, it fails because it is an object rather than a string. I think if we had a second match clause that let us pass through objects (possibly also arrays?) it would help eg:

match value { Value::String(text) => { let json = match serde_json::fromstr(text) { Err() => { return Err(RunnerError::new( source_info, RunnerErrorKind::QueryInvalidJson, false, )); } Ok(v) => v, }; eval_jsonpath_json(&json, expr, variables) } Value::Object => { eval_jsonpath_json(&value, expr, variables) } v => { let kind = RunnerErrorKind::FilterInvalidInput(v._type()); Err(RunnerError::new(source_info, kind, assert)) } }

jcamiel commented 2 weeks ago

Hi @luke-pp

It sounds reasonable, don't know if it's easy to implement @fabricereix but the features seems "natural" and user can expect it to work.

fabricereix commented 2 weeks ago

Hello, Yes, we could treat the Hurl Object as a JSON Object implicitly and apply jsonpath on it. We will have to convert each of its value to a JSON entity. We could also make it fail if the conversion does not make sense.

jcamiel commented 2 weeks ago

Just changed it from "bug" to "enhancement"