scylladb / seastar

High performance server-side application framework
http://seastar.io
Apache License 2.0
8.39k stars 1.55k forks source link

Abandoned generator leaks memory #2190

Open xemul opened 7 months ago

xemul commented 7 months ago

Consider the following test case (can be applied to tests/unit/coroutines_test.cc)

#include <seastar/core/queue.hh>

coroutine::experimental::generator<int> my_generator() {
    auto q = make_shared<queue<std::optional<int>>>(16);
    auto f = q->push_eventually(1).then([] {
        return sleep(1s);
    }).then([q] {
        return q->push_eventually({});
    });

    while (auto i = co_await q->pop_eventually()) {
        co_yield *i;
    }

    co_await std::move(f);
}

SEASTAR_TEST_CASE(test_early_abort_generator)
{
    auto gen = my_generator();
    std::optional<int> i = co_await gen();
    fmt::print("{}\n", i.value_or(42));
}

What it does is creates a my_generator, then co-awaits a single result out of it, then exists and (implicitly) destroys the generator. Generator itself, in turn, wants to generate two values, not one. But being destroyed early it leaks memory, reported in debug mode

=================================================================
==6430==ERROR: LeakSanitizer: detected memory leaks

Indirect leak of 168 byte(s) in 1 object(s) allocated from:
    #0 0x582641 in operator new(unsigned long) (/home/xemul/src/seastar/build/debug/tests/unit/coroutines_test+0x582641) (BuildId: 7560442d38b7f9121245a32bf098080d3a5f56a1)
    #1 0x830a8c in seastar::shared_ptr<seastar::queue<std::optional<int>>> seastar::shared_ptr_make_helper<seastar::queue<std::optional<int>>, false>::make<int>(int&&) /home/xemul/src/seastar/include/seastar/core/shared_ptr.hh:665:30
    #2 0x7a5616 in seastar::shared_ptr<seastar::queue<std::optional<int>>> seastar::make_shared<seastar::queue<std::optional<int>>, int>(int&&) /home/xemul/src/seastar/include/seastar/core/shared_ptr.hh:684:12
    #3 0x70497e in my_generator() (.resume) /home/xemul/src/seastar/tests/unit/coroutines_test.cc:556:14

line 556 here is the auto q = make_shared<queue<std::optional<int>>>(16); one

Indirect leak of 136 byte(s) in 1 object(s) allocated from:
    #0 0x582641 in operator new(unsigned long) (/home/xemul/src/seastar/build/debug/tests/unit/coroutines_test+0x582641) (BuildId: 7560442d38b7f9121245a32bf098080d3a5f56a1)
    #1 0x837e7f in seastar::future<void> seastar::sleep<std::chrono::_V2::steady_clock, long, std::ratio<1l, 1l>>(std::chrono::duration<long, std::ratio<1l, 1l>>) /home/xemul/src/seastar/include/seastar/core/sleep.hh:58:18
    #2 0x62182a in my_generator()::$_0::operator()() const /home/xemul/src/seastar/tests/unit/coroutines_test.cc:558:16

line 558 is the return sleep(1s); one

and few more

Making the test do co_await gen() in a loop until the generator drains itself solves leaks

xemul commented 7 months ago

Cc @tchaikov