danielaparker / jsoncons

A C++, header-only library for constructing JSON and JSON-like data formats, with JSON Pointer, JSON Patch, JSON Schema, JSONPath, JMESPath, CSV, MessagePack, CBOR, BSON, UBJSON
https://danielaparker.github.io/jsoncons
Other
726 stars 164 forks source link

`ojson jmespath::search()` heap use after free #471

Closed mtmorgan closed 11 months ago

mtmorgan commented 11 months ago

Describe the bug

This program queries a JSON document with JMESPath, using the 'ojson' type

#include <jsoncons/json.hpp>
#include <jsoncons_ext/jmespath/jmespath.hpp>

// for brevity
using jsoncons::ojson; 
namespace jmespath = jsoncons::jmespath;

int main() 
{
    std::string jtext = R"(
    {
      "locations": [
        {"name": "Seattle", "state": "WA"},
        {"name": "New York", "state": "NY"},
        {"name": "Bellevue", "state": "WA"},
        {"name": "Olympia", "state": "WA"}
      ]
    }        
    )";

    std::string expr = R"(
    {
        name: locations[].name,
        state: locations[].state
    }
    )";

    ojson doc = ojson::parse(jtext);

    ojson result = jmespath::search(doc, expr);

    std::cout << pretty_print(result) << "\n\n";
}

Compiling with 'address sanitizer'

g++ -std=gnu++17 -g -fsanitize=address -fno-omit-frame-pointer jmespath.cpp

indicates a memory access violation

./a.out
=================================================================
==22973==ERROR: AddressSanitizer: heap-use-after-free on address 0x0001033005d0 at pc 0x000100294628 bp 0x00016fb762c0 sp 0x00016fb762b8
READ of size 1 at 0x0001033005d0 thread T0
    #0 0x100294624 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::storage_kind() const basic_json.hpp:2844
    #1 0x100296834 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::size() const basic_json.hpp:2898
    #2 0x1002968b4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::size() const basic_json.hpp:2907
    #3 0x1002ac9b8 in jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>::flatten_and_destroy() json_object.hpp:1750
    #4 0x1002ac6d4 in jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>::~order_preserving_json_object() json_object.hpp:1158
    #5 0x1002ac6a8 in jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>::~order_preserving_json_object() json_object.hpp:1157
    #6 0x1002ac680 in std::__1::allocator<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>>::destroy[abi:v160006](jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>*) allocator.h:173
    #7 0x1002ac658 in void std::__1::allocator_traits<std::__1::allocator<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>>>::destroy[abi:v160006]<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>, void>(std::__1::allocator<jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>>&, jsoncons::order_preserving_json_object<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::vector>*) allocator_traits.h:315
    #8 0x1002ac534 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage::destroy() basic_json.hpp:1198
    #9 0x1002ac344 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage::~object_storage() basic_json.hpp:1139
    #10 0x1002ac2bc in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage::~object_storage() basic_json.hpp:1136
    #11 0x1002946d8 in void jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::destroy_var<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::object_storage>() basic_json.hpp:2244
    #12 0x100294590 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::destroy() basic_json.hpp:2228
    #13 0x1002944ec in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::~basic_json() basic_json.hpp:3822
    #14 0x10028d414 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::~basic_json() basic_json.hpp:3821
    #15 0x10037aaa0 in std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>::operator()[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) const unique_ptr.h:65
    #16 0x10037aa5c in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::reset[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) unique_ptr.h:297
    #17 0x10037a970 in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #18 0x100374dac in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #19 0x100379f74 in std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>::destroy[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator.h:173
    #20 0x100379cec in void std::__1::allocator_traits<std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::destroy[abi:v160006]<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, void>(std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>&, std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator_traits.h:315
    #21 0x10044db48 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__base_destruct_at_end[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) vector:836
    #22 0x10044da94 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__clear[abi:v160006]() vector:830
    #23 0x10044d8ac in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__destroy_vector::operator()[abi:v160006]() vector:446
    #24 0x10044d664 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #25 0x10044d51c in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #26 0x10044d4f0 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:390
    #27 0x10044c608 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:389
    #28 0x10034f3e8 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::jmespath_expression::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::error_code&) jmespath.hpp:3381
    #29 0x10028cde4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> jsoncons::jmespath::search<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::string_view_type const&) jmespath.hpp:5171
    #30 0x10028bbc4 in main jmespath.cpp:30
    #31 0x18924d0dc  (<unknown module>)

0x0001033005d0 is located 0 bytes inside of 16-byte region [0x0001033005d0,0x0001033005e0)
freed by thread T0 here:
    #0 0x100fa152c in wrap__ZdlPv+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x6152c)
    #1 0x10037aaa8 in std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>::operator()[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) const unique_ptr.h:65
    #2 0x10037aa5c in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::reset[abi:v160006](jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>*) unique_ptr.h:297
    #3 0x10037a970 in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #4 0x100374dac in std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>::~unique_ptr[abi:v160006]() unique_ptr.h:263
    #5 0x100379f74 in std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>::destroy[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator.h:173
    #6 0x100379cec in void std::__1::allocator_traits<std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::destroy[abi:v160006]<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, void>(std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>&, std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) allocator_traits.h:315
    #7 0x10044db48 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__base_destruct_at_end[abi:v160006](std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>*) vector:836
    #8 0x10044da94 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__clear[abi:v160006]() vector:830
    #9 0x10044d8ac in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::__destroy_vector::operator()[abi:v160006]() vector:446
    #10 0x10044d664 in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #11 0x10044d51c in std::__1::vector<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>, std::__1::allocator<std::__1::unique_ptr<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, std::__1::default_delete<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>>>>::~vector[abi:v160006]() vector:456
    #12 0x10044d4f0 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:390
    #13 0x10044c608 in jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::~dynamic_resources() jmespath.hpp:389
    #14 0x10034f3e8 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::jmespath_expression::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::error_code&) jmespath.hpp:3381
    #15 0x10028cde4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> jsoncons::jmespath::search<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::string_view_type const&) jmespath.hpp:5171
    #16 0x10028bbc4 in main jmespath.cpp:30
    #17 0x18924d0dc  (<unknown module>)

previously allocated by thread T0 here:
    #0 0x100fa10ec in wrap__Znwm+0x74 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x610ec)
    #1 0x100374aa4 in std::__1::__unique_if<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>::__unique_single std::__1::make_unique[abi:v160006]<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::json_array_arg_t const&>(jsoncons::json_array_arg_t const&) unique_ptr.h:686
    #2 0x100371da0 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>* jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::create_json<jsoncons::json_array_arg_t const&>(jsoncons::json_array_arg_t const&) jmespath.hpp:455
    #3 0x100444100 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::flatten_projection::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) const jmespath.hpp:3013
    #4 0x1003732e4 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::evaluate_tokens(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::vector<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token, std::__1::allocator<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) jmespath.hpp:2318
    #5 0x1003a0cbc in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::multi_select_hash::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) const jmespath.hpp:3141
    #6 0x1003732e4 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::evaluate_tokens(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::vector<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token, std::__1::allocator<jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::token>> const&, jsoncons::jmespath::detail::dynamic_resources<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>&, std::__1::error_code&) jmespath.hpp:2318
    #7 0x10034f3c8 in jsoncons::jmespath::detail::jmespath_evaluator<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&>::jmespath_expression::evaluate(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, std::__1::error_code&) jmespath.hpp:3380
    #8 0x10028cde4 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> jsoncons::jmespath::search<jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>>(jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>> const&, jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::string_view_type const&) jmespath.hpp:5171
    #9 0x10028bbc4 in main jmespath.cpp:30
    #10 0x18924d0dc  (<unknown module>)

SUMMARY: AddressSanitizer: heap-use-after-free basic_json.hpp:2844 in jsoncons::basic_json<char, jsoncons::order_preserving_policy, std::__1::allocator<char>>::storage_kind() const
Shadow bytes around the buggy address:
  0x000103300300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x000103300500: fa fa fa fa fa fa fd fd fa fa fd fd fa fa fd fd
=>0x000103300580: fa fa fd fd fa fa fd fd fa fa[fd]fd fa fa fd fd
  0x000103300600: fa fa fd fa fa fa 00 00 fa fa fd fd fa fa 00 fa
  0x000103300680: fa fa 00 fa fa fa fc fc fa fa fd fa fa fa fd fa
  0x000103300700: fa fa 00 00 fa fa 00 04 fa fa 00 fa fa fa 00 07
  0x000103300780: fa fa fd fd fa fa fd fd fa fa fa fa fa fa fa fa
  0x000103300800: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==22973==ABORTING

What compiler, architecture, and operating system?

$ g++ --version
Apple clang version 15.0.0 (clang-1500.0.40.1)
Target: arm64-apple-darwin23.1.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

What jsoncons library version?

mtmorgan commented 11 months ago

I am sure that this is not the correct fix, but the following removes the segfault by forcing dynamic_storage to be free'd early.

jsoncons/include master$ git diff
diff --git a/include/jsoncons_ext/jmespath/jmespath.hpp b/include/jsoncons_ext/jmespath/jmespath.hpp
index a091085b6..25931632c 100644
--- a/include/jsoncons_ext/jmespath/jmespath.hpp
+++ b/include/jsoncons_ext/jmespath/jmespath.hpp
@@ -3376,8 +3376,10 @@ namespace jmespath {
                 {
                     return Json::null();
                 }
-                dynamic_resources<Json,JsonReference> dynamic_storage;
-                return deep_copy(*evaluate_tokens(doc, output_stack_, dynamic_storage, ec));
+                dynamic_resources<Json,JsonReference>* dynamic_storage = new dynamic_resources<Json,JsonReference>();
+                Json copy = deep_copy(*evaluate_tokens(doc, output_stack_, *dynamic_storage, ec));
+                free(dynamic_storage);
+                return copy;
             }

             static jmespath_expression compile(const string_view_type& expr)
danielaparker commented 11 months ago

I am sure that this is not the correct fix, but the following removes the segfault by forcing dynamic_storage to be free'd early.

I'll have to study this a bit more, but could you also try

    Json copy;
    {
          dynamic_resources<Json,JsonReference> dynamic_storage;
          copy = deep_copy(*evaluate_tokens(doc, output_stack_, *dynamic_storage, ec);
    }
    return copy;

which should have the same affect?

Thanks, Daniel

mtmorgan commented 11 months ago

That variant does not help. Actually, I used the following, without a * before dynamic_storage and with a second ) after ec in the deep_copy() line, which I think is what you meant...

    Json copy;
    {
          dynamic_resources<Json,JsonReference> dynamic_storage;
          copy = deep_copy(*evaluate_tokens(doc, output_stack_, dynamic_storage, ec));
    }
    return copy;
danielaparker commented 11 months ago

Okay, I think if you changed

 free(dynamic_storage);

to

delete dynamic_storage;

in your variant, you'd see the same result as with my variant. That shouldn't happen, I'll need to investigate.

mtmorgan commented 11 months ago

yes, changing free() to delete in my variant also breaks the 'fix' and the heap use after free error returns.

danielaparker commented 11 months ago

Could you check if you still see the segfault with the code on master?

mtmorgan commented 11 months ago

Success!

danielaparker commented 11 months ago

Good. I think this is just another case of an over eager sanitizer. The "fix" was a work around that had no implications for heap memory access.