danielaparker / jsoncons

A C++, header-only library for constructing JSON and JSON-like data formats, with JSON Pointer, JSON Patch, JSON Schema, JSONPath, JMESPath, CSV, MessagePack, CBOR, BSON, UBJSON
https://danielaparker.github.io/jsoncons
Other
726 stars 164 forks source link

Filters are not working correctly in version 169. #413

Closed dnj12345 closed 1 year ago

dnj12345 commented 1 year ago

I recently upgraded my project to use the latest jsoncons version v0.169.0. Since then JSON path filter expressions are not working correctly. The same JSON path expression with filter works fine with jsoncons version v0.106.0. I've tested my expressions at various websites with mixed results: javainuse.com/jsonpath (works), jsonpath.com (fails), jsonquerytools.com (fails)

------------------------ jsoncons v0.106.0 test output --------------------------------------------
[root@7c0300cc92e8 jsoncons]# ./test.106 "$..bicycle.wheels[?(@.type == 'tubular')]" test.json
jsoncons-version: 0.106.0
json:
{
    "store": {
        "bicycle": {
            "color": "red",
            "price": 19.95,
            "wheels": {
                "size_inches": 26,
                "type": "tubular"
            }
        },
        "books": [
            {
                "author": "Nigel Rees",
                "category": "reference",
                "price": 8.95,
                "title": "Sayings of the Century"
            },
            {
                "author": "Evelyn Waugh",
                "category": "fiction",
                "price": 12.99,
                "title": "Sword of Honour"
            }
        ],
        "num_categories": 20,
        "num_items": 1234
    }
}
expr: $..bicycle.wheels[?(@.type == 'tubular')]
result: [{"size_inches":26,"type":"tubular"}]
length: 1

--------------------------------------------------------------------------------------------------

As you can see, the expression yields results. Here are the results for v0.169.0

------------------------ jsoncons v0.169.0 test output --------------------------------------------
[root@0242845ec171 jsoncons]# ./test.169 "$..bicycle.wheels[?(@.type=='tubular')]" test.json
jsoncons-version: 0.169.0
json:
{
    "store": {
        "bicycle": {
            "color": "red",
            "price": 19.95,
            "wheels": {
                "size_inches": 26,
                "type": "tubular"
            }
        },
        "books": [
            {
                "author": "Nigel Rees",
                "category": "reference",
                "price": 8.95,
                "title": "Sayings of the Century"
            },
            {
                "author": "Evelyn Waugh",
                "category": "fiction",
                "price": 12.99,
                "title": "Sword of Honour"
            }
        ],
        "num_categories": 20,
        "num_items": 1234
    }
}
expr: $..bicycle.wheels[?(@.type=='tubular')]
result: []
length: 0

----------------------------------------------------------------------------------------------

Here's my test program.

#include <string>
#include <fstream>
#include <jsoncons/json.hpp>
#include <jsoncons_ext/jsonpath/json_query.hpp>

using namespace jsoncons;
using namespace jsoncons::jsonpath;

void
test_json_expr(std::string& expr, std::string& in_file)
{
  json j;
  if (in_file.empty()) {
    try
    {
      j = json::parse(R"(
                      {"store":
                      {"book": [
                      {"category": "reference",
                      "author": "Margaret Weis",
                      "title": "Dragonlance Series",
                      "price": 31.96},
                      {"category": "reference",
                      "author": "Brent Weeks",
                      "title": "Night Angel Trilogy",
                      "price": 14.70
                      }]}}
                      )");
    }
    catch (const std::exception& e)
    {
      std::cout << e.what() << std::endl;
    }
  } else {
        std::ifstream is(in_file);
        is >> j;
  }

  std::cout << "json:\n" << pretty_print(j) << std::endl;

  json result1;
  try {
    result1 = json_query(j, expr);
  }
  catch (std::exception &e) {
    std::cout << "json query exception: " << e.what() << "\n";
    std::exit(-1);
  }

  std::cout << "expr: " << expr << std::endl;
  std::cout << "result: " << result1 << std::endl;
  std::cout << "length: " << result1.size() << std::endl;
}

int
main(int argc, char *argv[])
{
  if (argc < 2) {
    std::cout << "Usage: " << argv[0] << "[<expr>]\n";
    std::exit(-1);
  }

  std::string expr;
  if (argc >= 2) {
    expr = std::string(argv[1]);
  }

  std::string in_file;
  if (argc == 3) {
    in_file = std::string(argv[2]);
  }

  std::cout << "jsoncons-version: " << jsoncons::version() << std::endl;
  test_json_expr(expr, in_file);
  std::cout << std::endl;
}

The two versions were tested on two different containers because of the upgrade to the container and compilers etc. Could not build older versions with the newer compiler etc.

v0.169.0:

v0.106.0:

economicseries commented 1 year ago

Different JSONPath implementations have different understandings of applying a filter to an object.

(1) Some don't allow it at all, only to arrays (2) Some apply the filter to the object itself (following Jayway, jsoncons did this up until version 0.161.0) (3) Some apply the filter to the value part of the object's name value pairs (jsoncons does this since version 0.161.0.)

For the three websites that you checked, the first exhibits the behavior described in (2), and the second and third exhibit the behavior described in (3).

To make your query work with the second and third websites (and jsoncons since 0.161.0), rewrite it as

$..bicycle[?(@.type == 'tubular')]
dnj12345 commented 1 year ago

ok. Thanks for the clarification.

Not sure if it makes sense to add a note in the documentation.