Stack overflow with recursive refs

tyler92 commented 1 month ago

Example:

[
    {
        "description": "recursive ref",
        "schema": {
            "items": [
                {"$ref": "#/items/0"}
            ]
        },
        "tests": [ ]
    }
]

Address sanitizer report:

AddressSanitizer:DEADLYSIGNAL
=================================================================
==1872324==ERROR: AddressSanitizer: stack-overflow on address 0x7ffe150bea88 (pc 0x55dfc89a4c0f bp 0x7ffe150bf2d0 sp 0x7ffe150bea90 T0)
    #0 0x55dfc89a4c0f in __asan_memcpy (valijson/myfuzzing/prefix/bin/fuzzer+0x16ec0f) (BuildId: fb50edf90e4b5e099f651da860e13ff632c933ef)
    #1 0x55dfc8aaec60 in valijson::adapters::BasicAdapter<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonArray<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonObjectMember<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonObject<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonValue<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>::BasicAdapter(valijson::adapters::BasicAdapter<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonArray<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonObjectMember<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonObject<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>, valijson::adapters::GenericRapidJsonValue<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>> const&) valijson/myfuzzing/cmake-build/../../include/valijson/internal/basic_adapter.hpp:83:7
    #2 0x55dfc8aaeb5a in valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>::GenericRapidJsonAdapter(valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&) valijson/myfuzzing/cmake-build/../../include/valijson/adapters/rapidjson_adapter.hpp:611:7
    #3 0x55dfc8bc2c1c in valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> valijson::internal::json_pointer::resolveJsonPointer<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>(valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>) valijson/myfuzzing/cmake-build/../../include/valijson/internal/json_pointer.hpp:166:16
    #4 0x55dfc8bc3a83 in valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> valijson::internal::json_pointer::resolveJsonPointer<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>(valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>) valijson/myfuzzing/cmake-build/../../include/valijson/internal/json_pointer.hpp:220:20
    #5 0x55dfc8bc4110 in valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> valijson::internal::json_pointer::resolveJsonPointer<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>(valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, __gnu_cxx::__normal_iterator<char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>) valijson/myfuzzing/cmake-build/../../include/valijson/internal/json_pointer.hpp:244:16
    #6 0x55dfc8b5a28a in valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> valijson::internal::json_pointer::resolveJsonPointer<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>(valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&) valijson/myfuzzing/cmake-build/../../include/valijson/internal/json_pointer.hpp:266:12
    #7 0x55dfc8b80507 in valijson::Subschema const* valijson::SchemaParser::makeOrReuseSchema<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>(valijson::Schema&, valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, valijson::SchemaParser::FunctionPtrs<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>::FetchDoc, valijson::Subschema const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const*, valijson::SchemaParser::DocumentCache<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>::Type&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, valijson::Subschema const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const, valijson::Subschema const*>>>&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>&) valijson/schema_parser.hpp:489:17
    #8 0x55dfc8b80610 in valijson::Subschema const* valijson::SchemaParser::makeOrReuseSchema<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>(valijson::Schema&, valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, valijson::SchemaParser::FunctionPtrs<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>::FetchDoc, valijson::Subschema const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const*, valijson::SchemaParser::DocumentCache<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>::Type&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, valijson::Subschema const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const, valijson::Subschema const*>>>&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>&) valijson/schema_parser.hpp:496:16
    #9 0x55dfc8b80610 in valijson::Subschema const* valijson::SchemaParser::makeOrReuseSchema<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>(valijson::Schema&, valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>> const&, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const&, valijson::SchemaParser::FunctionPtrs<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>::FetchDoc, valijson::Subschema const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const*, valijson::SchemaParser::DocumentCache<valijson::adapters::GenericRapidJsonAdapter<rapidjson::GenericValue<rapidjson::UTF8<char>, rapidjson::MemoryPoolAllocator<rapidjson::CrtAllocator>>>>::Type&, std::map<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, valijson::Subschema const*, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>, std::allocator<std::pair<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>> const, valijson::Subschema const*>>>&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char>>>>&) valijson/schema_parser.hpp:496:16
    ...

tristanpenman commented 1 month ago

Awesome work @tyler92 - all of your PRs have been merged. Thanks for the contribution! 🔥

Now I'll just let the dust settle for a few days before cutting a new release. If there are any issues with the fuzzing changes downstream, we should know with a day or two.

tyler92 commented 1 month ago

Probably I created too many PRs :slightly_smiling_face: . This issue is not solved and it's better to reopen it. Just in case, here is an overview of my PRs/issues:

fix for u8_strlen: https://github.com/tristanpenman/valijson/pull/197 (merged)
fix for division by zero: https://github.com/tristanpenman/valijson/pull/198 (merged)
fix for stack overflow for invalid references: https://github.com/tristanpenman/valijson/pull/199 (merged)
fuzzing improvements: https://github.com/tristanpenman/valijson/pull/201 (merged)
issue for stack overflow for recursive references: https://github.com/tristanpenman/valijson/issues/200 (this one)

I think it's OK that fuzzing merged, in the worst case you will receive an e-mail from oss-fuzz about that bug later

tristanpenman commented 1 month ago

My bad 🤦 I didn't look at the description closely enough and mixed it up with the PR for fixing unresolved references.

I'll leave this open until the issue is correctly resolved.

tristanpenman / valijson

Stack overflow with recursive refs #200