Unpack example from quickstart segfaults

[Kobzol]

Describe the bug When I try to run the "Streaming feature" example from the C++ quickstart, the program segfaults.

Using the cpp_master branch (commit 2987022c10ac066b55c6e554369eab92cdebe506). It works fine with the cpp-6.1.0 tag.

To Reproduce Test file (test.cpp):

#include <msgpack.hpp>
#include <iostream>
#include <string>

int main() {
        // serializes multiple objects using msgpack::packer.
        msgpack::sbuffer buffer;

        msgpack::packer<msgpack::sbuffer> pk(&buffer);
        pk.pack(std::string("Log message ... 1"));
        pk.pack(std::string("Log message ... 2"));
        pk.pack(std::string("Log message ... 3"));

        // deserializes these objects using msgpack::unpacker.
        msgpack::unpacker pac;

        // feeds the buffer.
        pac.reserve_buffer(buffer.size());
        memcpy(pac.buffer(), buffer.data(), buffer.size());
        pac.buffer_consumed(buffer.size());

        // now starts streaming deserialization.
        msgpack::object_handle oh;
        while(pac.next(oh)) {
            std::cout << oh.get() << std::endl;
        }

        // results:
        // $ g++ -Ipath_to_msgpack/include stream.cc -o stream
        // $ ./stream
        // "Log message ... 1"
        // "Log message ... 2"
        // "Log message ... 3"
}

$ git checkout 2987022c10ac066b55c6e554369eab92cdebe506
$ g++ test.cpp -O2 -otest
$ ./test
"Log message ... 1"
Segmentation fault (core dumped)

Using g++ 11.4.0 on Ubuntu 22.04. It works without optimizations, so it's most probably some UB/memory error. Works with clang 14.0.0 (both with and without optimizations).

Here is the Valgrind output:

Memcheck output

``` ==176304== Invalid read of size 8 ==176304== at 0x10AA68: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b108 is 8 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10AA07: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid read of size 8 ==176304== at 0x10AA6C: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b100 is 0 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10AA07: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid read of size 8 ==176304== at 0x10AA72: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b110 is 16 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10AA07: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid free() / delete / delete[] / realloc() ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10AA7F: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b100 is 0 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10AA07: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== "Log message ... 2" ==176304== Invalid read of size 1 ==176304== at 0x10B515: msgpack::v1::object_stringize_visitor::visit_str(char const*, unsigned int) (in /tmp/msgpack-c/example/test) ==176304== by 0x10D3CE: void msgpack::v1::object_parser::parse(msgpack::v1::object_stringize_visitor&) (in /tmp/msgpack-c/example/test) ==176304== by 0x10AB6D: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4dfaf49 is 41 bytes inside a block of size 65,536 free'd ==176304== at 0x484BB2C: free (vg_replace_malloc.c:987) ==176304== by 0x10AA6E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x484880F: malloc (vg_replace_malloc.c:444) ==176304== by 0x10CD2C: msgpack::v2::unpacker::unpacker(bool (*)(msgpack::v1::type::object_type, unsigned long, void*), void*, unsigned long, msgpack::v1::unpack_limit const&) (in /tmp/msgpack-c/example/test) ==176304== by 0x10A7B5: main (in /tmp/msgpack-c/example/test) ==176304== "Log message ... 3" ==176304== Invalid read of size 4 ==176304== at 0x10B444: msgpack::v1::detail::decr_count(void*) (in /tmp/msgpack-c/example/test) ==176304== by 0x10A88D: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4dfaf20 is 0 bytes inside a block of size 65,536 free'd ==176304== at 0x484BB2C: free (vg_replace_malloc.c:987) ==176304== by 0x10AA6E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x484880F: malloc (vg_replace_malloc.c:444) ==176304== by 0x10CD2C: msgpack::v2::unpacker::unpacker(bool (*)(msgpack::v1::type::object_type, unsigned long, void*), void*, unsigned long, msgpack::v1::unpack_limit const&) (in /tmp/msgpack-c/example/test) ==176304== by 0x10A7B5: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid read of size 8 ==176304== at 0x10A8F0: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b6c8 is 8 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10A89E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid read of size 8 ==176304== at 0x10A8F4: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b6c0 is 0 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10A89E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid read of size 4 ==176304== at 0x10B444: msgpack::v1::detail::decr_count(void*) (in /tmp/msgpack-c/example/test) ==176304== by 0x10A8F5: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4dfaf20 is 0 bytes inside a block of size 65,536 free'd ==176304== at 0x484BB2C: free (vg_replace_malloc.c:987) ==176304== by 0x10AA6E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x484880F: malloc (vg_replace_malloc.c:444) ==176304== by 0x10CD2C: msgpack::v2::unpacker::unpacker(bool (*)(msgpack::v1::type::object_type, unsigned long, void*), void*, unsigned long, msgpack::v1::unpack_limit const&) (in /tmp/msgpack-c/example/test) ==176304== by 0x10A7B5: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid read of size 8 ==176304== at 0x10A8F9: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b6d0 is 16 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10A89E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid free() / delete / delete[] / realloc() ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10A906: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4e0b6c0 is 0 bytes inside a block of size 24 free'd ==176304== at 0x484C93D: operator delete(void*, unsigned long) (vg_replace_malloc.c:1103) ==176304== by 0x10A89E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x4848F95: operator new(unsigned long) (vg_replace_malloc.c:485) ==176304== by 0x10AC50: main (in /tmp/msgpack-c/example/test) ==176304== ==176304== Invalid read of size 4 ==176304== at 0x10C538: msgpack::v2::unpacker::~unpacker() (in /tmp/msgpack-c/example/test) ==176304== by 0x10A937: main (in /tmp/msgpack-c/example/test) ==176304== Address 0x4dfaf20 is 0 bytes inside a block of size 65,536 free'd ==176304== at 0x484BB2C: free (vg_replace_malloc.c:987) ==176304== by 0x10AA6E: main (in /tmp/msgpack-c/example/test) ==176304== Block was alloc'd at ==176304== at 0x484880F: malloc (vg_replace_malloc.c:444) ==176304== by 0x10CD2C: msgpack::v2::unpacker::unpacker(bool (*)(msgpack::v1::type::object_type, unsigned long, void*), void*, unsigned long, msgpack::v1::unpack_limit const&) (in /tmp/msgpack-c/example/test) ==176304== by 0x10A7B5: main (in /tmp/msgpack-c/example/test) ```

Expected behavior It should not crash.

[redboltz]

@Kobzol ,Thank you for reporting the issue. I reproduced the valgrind result. Is seems that the issue is caused by #1096 (including #1095).

Could you try 37fcaa1264746456c14aec08bf6efc8d1ab06f7b (Just before commit merging #1096). On my environment, it works fine. @Kobzol , could you try it? If it would work well too, I will revert #1096.

@Arenoros, could you debug #1096 ? If you would fixed the issue, please create a new PR.

[Kobzol]

I can confirm that with this commit, it doesn't segfault, and Valgrind doesn't report any errors.

[redboltz]

Fixed by　#1101