danielaparker / jsoncons

A C++, header-only library for constructing JSON and JSON-like data formats, with JSON Pointer, JSON Patch, JSON Schema, JSONPath, JMESPath, CSV, MessagePack, CBOR, BSON, UBJSON
https://danielaparker.github.io/jsoncons
Other
697 stars 160 forks source link

`ojson jmespath::search()` heap use after free #471

Closed mtmorgan closed 8 months ago

mtmorgan commented 8 months ago

Describe the bug

This program queries a JSON document with JMESPath, using the 'ojson' type

#include <jsoncons/json.hpp>
#include <jsoncons_ext/jmespath/jmespath.hpp>

// for brevity
using jsoncons::ojson; 
namespace jmespath = jsoncons::jmespath;

int main() 
{
    std::string jtext = R"(
    {
      "locations": [
        {"name": "Seattle", "state": "WA"},
        {"name": "New York", "state": "NY"},
        {"name": "Bellevue", "state": "WA"},
        {"name": "Olympia", "state": "WA"}
      ]
    }        
    )";

    std::string expr = R"(
    {
        name: locations[].name,
        state: locations[].state
    }
    )";

    ojson doc = ojson::parse(jtext);

    ojson result = jmespath::search(doc, expr);

    std::cout << pretty_print(result) << "\n\n";
}

Compiling with 'address sanitizer'

g++ -std=gnu++17 -g -fsanitize=address -fno-omit-frame-pointer jmespath.cpp

indicates a memory access violation

./a.out
=================================================================
==22973==ERROR: AddressSanitizer: heap-use-after-free on address 0x0001033005d0 at pc 0x000100294628 bp 0x00016fb762c0 sp 0x00016fb762b8
READ of size 1 at 0x0001033005d0 thread T0
    #0 0x100294624 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::storage_kind() const basic_json.hpp:2844
    #1 0x100296834 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::size() const basic_json.hpp:2898
    #2 0x1002968b4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::size() const basic_json.hpp:2907
    #3 0x1002ac9b8 in jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>::flatten_and_destroy() json_object.hpp:1750
    #4 0x1002ac6d4 in jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>::~order_preserving_json_object() json_object.hpp:1158
    #5 0x1002ac6a8 in jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>::~order_preserving_json_object() json_object.hpp:1157
    #6 0x1002ac680 in std::__1::allocator<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>>::destroy[abi:v160006](jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>*) allocator.h:173
    #7 0x1002ac658 in void std::__1::allocator_traits<std::__1::allocator<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>>>::destroy[abi:v160006]<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>, void>(std::__1::allocator<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>>&, jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>*) allocator_traits.h:315
    #8 0x1002ac534 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage::destroy() basic_json.hpp:1198
    #9 0x1002ac344 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage::~object_storage() basic_json.hpp:1139
    #10 0x1002ac2bc in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage::~object_storage() basic_json.hpp:1136
    #11 0x1002946d8 in void jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::destroy_var<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage>() basic_json.hpp:2244
    #12 0x100294590 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::destroy() basic_json.hpp:2228
    #13 0x1002944ec in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::~basic_json() basic_json.hpp:3822
    #14 0x10028d414 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::~basic_json() basic_json.hpp:3821
    #15 0x10037aaa0 in std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>::operator()[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) const unique_ptr.h:65
    #16 0x10037aa5c in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::reset[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) unique_ptr.h:297
    #17 0x10037a970 in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #18 0x100374dac in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #19 0x100379f74 in std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>::destroy[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator.h:173
    #20 0x100379cec in void std::__1::allocator_traits<std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::destroy[abi:v160006]<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, void>(std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>&, std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator_traits.h:315
    #21 0x10044db48 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__base_destruct_at_end[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) vector:836
    #22 0x10044da94 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__clear[abi:v160006]() vector:830
    #23 0x10044d8ac in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__destroy_vector::operator()[abi:v160006]() vector:446
    #24 0x10044d664 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #25 0x10044d51c in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #26 0x10044d4f0 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:390
    #27 0x10044c608 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:389
    #28 0x10034f3e8 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::jmespath_expression::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::error_code&) jmespath.hpp:3381
    #29 0x10028cde4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> jsoncons::jmespath::search<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::string_view_type const&) jmespath.hpp:5171
    #30 0x10028bbc4 in main jmespath.cpp:30
    #31 0x18924d0dc  (<unknown module>)

0x0001033005d0 is located 0 bytes inside of 16-byte region [0x0001033005d0,0x0001033005e0)
freed by thread T0 here:
    #0 0x100fa152c in wrap__ZdlPv+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x6152c)
    #1 0x10037aaa8 in std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>::operator()[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) const unique_ptr.h:65
    #2 0x10037aa5c in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::reset[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) unique_ptr.h:297
    #3 0x10037a970 in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #4 0x100374dac in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #5 0x100379f74 in std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>::destroy[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator.h:173
    #6 0x100379cec in void std::__1::allocator_traits<std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::destroy[abi:v160006]<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, void>(std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>&, std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator_traits.h:315
    #7 0x10044db48 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__base_destruct_at_end[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) vector:836
    #8 0x10044da94 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__clear[abi:v160006]() vector:830
    #9 0x10044d8ac in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__destroy_vector::operator()[abi:v160006]() vector:446
    #10 0x10044d664 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #11 0x10044d51c in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #12 0x10044d4f0 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:390
    #13 0x10044c608 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:389
    #14 0x10034f3e8 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::jmespath_expression::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::error_code&) jmespath.hpp:3381
    #15 0x10028cde4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> jsoncons::jmespath::search<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::string_view_type const&) jmespath.hpp:5171
    #16 0x10028bbc4 in main jmespath.cpp:30
    #17 0x18924d0dc  (<unknown module>)

previously allocated by thread T0 here:
    #0 0x100fa10ec in wrap__Znwm+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x610ec)
    #1 0x100374aa4 in std::__1::__unique_if<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>::__unique_single std::__1::make_unique[abi:v160006]<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::json_array_arg_t const&>(jsoncons::json_array_arg_t const&) unique_ptr.h:686
    #2 0x100371da0 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>* jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::create_json<jsoncons::json_array_arg_t const&>(jsoncons::json_array_arg_t const&) jmespath.hpp:455
    #3 0x100444100 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::flatten_projection::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) const jmespath.hpp:3013
    #4 0x1003732e4 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::evaluate_tokens(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::vector<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token, std::__1::allocator<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) jmespath.hpp:2318
    #5 0x1003a0cbc in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::multi_select_hash::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) const jmespath.hpp:3141
    #6 0x1003732e4 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::evaluate_tokens(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::vector<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token, std::__1::allocator<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) jmespath.hpp:2318
    #7 0x10034f3c8 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::jmespath_expression::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::error_code&) jmespath.hpp:3380
    #8 0x10028cde4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> jsoncons::jmespath::search<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::string_view_type const&) jmespath.hpp:5171
    #9 0x10028bbc4 in main jmespath.cpp:30
    #10 0x18924d0dc  (<unknown module>)

SUMMARY: AddressSanitizer: heap-use-after-free basic_json.hpp:2844 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::storage_kind() const
Shadow bytes around the buggy address:
  0x000103300300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300500: fa fa fa fa fa fa fd fd fa fa fd fd fa fa fd fd
=>0x000103300580: fa fa fd fd fa fa fd fd fa fa[fd]fd fa fa fd fd
  0x000103300600: fa fa fd fa fa fa 00 00 fa fa fd fd fa fa 00 fa
  0x000103300680: fa fa 00 fa fa fa fc fc fa fa fd fa fa fa fd fa
  0x000103300700: fa fa 00 00 fa fa 00 04 fa fa 00 fa fa fa 00 07
  0x000103300780: fa fa fd fd fa fa fd fd fa fa fa fa fa fa fa fa
  0x000103300800: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==22973==ABORTING

What compiler, architecture, and operating system?

$ g++ --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

What jsoncons library version?

mtmorgan commented 8 months ago

I am sure that this is not the correct fix, but the following removes the segfault by forcing dynamic_storage to be free'd early.

jsoncons/include master$ git diff
diff --git a/include/jsoncons_ext/jmespath/jmespath.hpp b/include/jsoncons_ext/jmespath/jmespath.hpp
index a091085b6..25931632c 100644
--- a/include/jsoncons_ext/jmespath/jmespath.hpp
+++ b/include/jsoncons_ext/jmespath/jmespath.hpp
@@ -3376,8 +3376,10 @@ namespace jmespath {
                 {
                     return Json::null();
                 }
-                dynamic_resources<Json,JsonReference> dynamic_storage;
-                return deep_copy(*evaluate_tokens(doc, output_stack_, dynamic_storage, ec));
+                dynamic_resources<Json,JsonReference>* dynamic_storage = new dynamic_resources<Json,JsonReference>();
+                Json copy = deep_copy(*evaluate_tokens(doc, output_stack_, *dynamic_storage, ec));
+                free(dynamic_storage);
+                return copy;
             }

             static jmespath_expression compile(const string_view_type& expr)
danielaparker commented 8 months ago

I am sure that this is not the correct fix, but the following removes the segfault by forcing dynamic_storage to be free'd early.

I'll have to study this a bit more, but could you also try

    Json copy;
    {
          dynamic_resources<Json,JsonReference> dynamic_storage;
          copy = deep_copy(*evaluate_tokens(doc, output_stack_, *dynamic_storage, ec);
    }
    return copy;

which should have the same affect?

Thanks, Daniel

mtmorgan commented 8 months ago

That variant does not help. Actually, I used the following, without a * before dynamic_storage and with a second ) after ec in the deep_copy() line, which I think is what you meant...

    Json copy;
    {
          dynamic_resources<Json,JsonReference> dynamic_storage;
          copy = deep_copy(*evaluate_tokens(doc, output_stack_, dynamic_storage, ec));
    }
    return copy;
danielaparker commented 8 months ago

Okay, I think if you changed

 free(dynamic_storage);

to

delete dynamic_storage;

in your variant, you'd see the same result as with my variant. That shouldn't happen, I'll need to investigate.

mtmorgan commented 8 months ago

yes, changing free() to delete in my variant also breaks the 'fix' and the heap use after free error returns.

danielaparker commented 8 months ago

Could you check if you still see the segfault with the code on master?

mtmorgan commented 8 months ago

Success!

danielaparker commented 8 months ago

Good. I think this is just another case of an over eager sanitizer. The "fix" was a work around that had no implications for heap memory access.