Orange-OpenSource / hurl

Hurl, run and test HTTP requests with plain text.
https://hurl.dev
Apache License 2.0
12.73k stars 477 forks source link

Improve JSONPath eval performance #3018

Open jcamiel opened 2 months ago

jcamiel commented 2 months ago

In packages/hurl/src/jsonpath/eval/query.rs:

impl Query {
    /// Eval a JSONPath `Query` for a `serde_json::Value` input.
    /// It returns an Option<`JsonResultPath`>.
    pub fn eval(&self, value: &serde_json::Value) -> Option<JsonpathResult> {
        let mut result = JsonpathResult::SingleEntry(value.clone());
        for selector in &self.selectors {
            match result.clone() {
                JsonpathResult::SingleEntry(value) => {
                    result = selector.eval(&value)?;
                }
                JsonpathResult::Collection(values) => {
                    let mut elements = vec![];
                    for value in values {
                        match selector.eval(&value)? {
                            JsonpathResult::SingleEntry(new_value) => {
                                elements.push(new_value);
                            }
                            JsonpathResult::Collection(mut new_values) => {
                                elements.append(&mut new_values);
                            }
                        }
                        result = JsonpathResult::Collection(elements.clone());
                    }
                }
            }
        }
        Some(result)
    }
}

The first thing we do is cloning the input value:

    pub fn eval(&self, value: &serde_json::Value) -> Option<JsonpathResult> {
        let mut result = JsonpathResult::SingleEntry(value.clone());

For every jsonpath query, the input value is the root document. For a big JSON response, it means cloning the response for every query, even if we need a small node. As we've serde_json::Value as input and a result containing serde_json::Value, maybe we can work exclusivley with references on the input.

Check https://github.com/Orange-OpenSource/hurl/blob/master/integration/hurl/tests_ok/parse_cache.hurl, we're evaluating ~80 JSONPath query on a 1.3M JSON document in 180 ms (at 380 ms):

Screenshot 2024-07-05 at 14 00 31