Open michaelsippel opened 10 months ago
There is still some bug that leaks memory at one place that we need to fix in this PR.
But a some quick preliminary benchmarks reveal that we might be on the right track here and we might be indeed limited by memory latency in the case of 64x64 tiles.
Rebased this branch and with the latest patches for ChunkedList, the memory-leaks previously observed on this branch are gone now.
After this PR is in the mainline I will add a clang format file and auto-format the code.
Rebased again, and I'm not able to break it so far, all tests look fine.
Only concern I have now is the use of offsetof
in https://github.com/ComputationalRadiationPhysics/redGrapes/blob/e5ea9e48e52217e100351dcd30ff4dea18f75380/redGrapes/task/property/graph.cpp#L28 where I get the warning that this is 'conditionally supported' , but it seems to work fine.
Ideally we would need offsetof
for base-classes.
This PR produces many address sanitizer issues.
=================================================================
1: ==46845==ERROR: LeakSanitizer: detected memory leaks
1:
1: Direct leak of 360 byte(s) in 3 object(s) allocated from:
1: #0 0x7fbdffab61e7 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
1: #1 0x559086696289 in redGrapes::Context::init(unsigned long, std::shared_ptr<redGrapes::scheduler::IScheduler>) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:132
1: #2 0x55908669647f in redGrapes::Context::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:140
1: #3 0x5590865d49d3 in redGrapes::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.hpp:122
1: #4 0x559086669d59 in test_worker_utilization(unsigned int) /home/widera/workspace/redGrapes/test/scheduler.cpp:20
1: #5 0x55908666a61b in CATCH2_INTERNAL_TEST_0 /home/widera/workspace/redGrapes/test/scheduler.cpp:52
1: #6 0x5590867a04da in Catch::TestInvokerAsFunction::invoke() const src/catch2/internal/catch_test_case_registry_impl.cpp:149
1: #7 0x559086782f9f in Catch::TestCaseHandle::invoke() const (/home/widera/workspace/buildRedGrapes/redGrapes_test+0x22bf9f)
1: #8 0x559086780717 in Catch::RunContext::invokeActiveTestCase() src/catch2/internal/catch_run_context.cpp:538
1: #9 0x55908677fed0 in Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/catch2/internal/catch_run_context.cpp:501
1: #10 0x55908677b7b0 in Catch::RunContext::runTest(Catch::TestCaseHandle const&) src/catch2/internal/catch_run_context.cpp:232
1: #11 0x5590866befbe in execute src/catch2/catch_session.cpp:110
1: #12 0x5590866c1f40 in Catch::Session::runInternal() src/catch2/catch_session.cpp:332
1: #13 0x5590866c1446 in Catch::Session::run() src/catch2/catch_session.cpp:263
1: #14 0x5590866bcfab in int Catch::Session::run<char>(int, char const* const*) src/catch2/../catch2/catch_session.hpp:41
1: #15 0x5590866bcd66 in main src/catch2/internal/catch_main.cpp:36
1: #16 0x7fbdff229d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
1:
1: Direct leak of 120 byte(s) in 1 object(s) allocated from:
1: #0 0x7fbdffab61e7 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
1: #1 0x559086696289 in redGrapes::Context::init(unsigned long, std::shared_ptr<redGrapes::scheduler::IScheduler>) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:132
1: #2 0x55908669647f in redGrapes::Context::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:140
1: #3 0x5590865d49d3 in redGrapes::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.hpp:122
1: #4 0x55908663131c in CATCH2_INTERNAL_TEST_0 /home/widera/workspace/redGrapes/test/resource_user.cpp:10
1: #5 0x5590867a04da in Catch::TestInvokerAsFunction::invoke() const src/catch2/internal/catch_test_case_registry_impl.cpp:149
1: #6 0x559086782f9f in Catch::TestCaseHandle::invoke() const (/home/widera/workspace/buildRedGrapes/redGrapes_test+0x22bf9f)
1: #7 0x559086780717 in Catch::RunContext::invokeActiveTestCase() src/catch2/internal/catch_run_context.cpp:538
1: #8 0x55908677fed0 in Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/catch2/internal/catch_run_context.cpp:501
1: #9 0x55908677b7b0 in Catch::RunContext::runTest(Catch::TestCaseHandle const&) src/catch2/internal/catch_run_context.cpp:232
1: #10 0x5590866befbe in execute src/catch2/catch_session.cpp:110
1: #11 0x5590866c1f40 in Catch::Session::runInternal() src/catch2/catch_session.cpp:332
1: #12 0x5590866c1446 in Catch::Session::run() src/catch2/catch_session.cpp:263
1: #13 0x5590866bcfab in int Catch::Session::run<char>(int, char const* const*) src/catch2/../catch2/catch_session.hpp:41
1: #14 0x5590866bcd66 in main src/catch2/internal/catch_main.cpp:36
1: #15 0x7fbdff229d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
1:
1: Direct leak of 120 byte(s) in 1 object(s) allocated from:
1: #0 0x7fbdffab61e7 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
1: #1 0x559086696289 in redGrapes::Context::init(unsigned long, std::shared_ptr<redGrapes::scheduler::IScheduler>) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:132
1: #2 0x55908669647f in redGrapes::Context::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:140
1: #3 0x5590865d49d3 in redGrapes::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.hpp:122
1: #4 0x5590865cb17c in CATCH2_INTERNAL_TEST_2 /home/widera/workspace/redGrapes/test/resource.cpp:60
1: #5 0x5590867a04da in Catch::TestInvokerAsFunction::invoke() const src/catch2/internal/catch_test_case_registry_impl.cpp:149
1: #6 0x559086782f9f in Catch::TestCaseHandle::invoke() const (/home/widera/workspace/buildRedGrapes/redGrapes_test+0x22bf9f)
1: #7 0x559086780717 in Catch::RunContext::invokeActiveTestCase() src/catch2/internal/catch_run_context.cpp:538
1: #8 0x55908677fed0 in Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/catch2/internal/catch_run_context.cpp:501
1: #9 0x55908677b7b0 in Catch::RunContext::runTest(Catch::TestCaseHandle const&) src/catch2/internal/catch_run_context.cpp:232
1: #10 0x5590866befbe in execute src/catch2/catch_session.cpp:110
1: #11 0x5590866c1f40 in Catch::Session::runInternal() src/catch2/catch_session.cpp:332
1: #12 0x5590866c1446 in Catch::Session::run() src/catch2/catch_session.cpp:263
1: #13 0x5590866bcfab in int Catch::Session::run<char>(int, char const* const*) src/catch2/../catch2/catch_session.hpp:41
1: #14 0x5590866bcd66 in main src/catch2/internal/catch_main.cpp:36
1: #15 0x7fbdff229d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
1:
1: Direct leak of 120 byte(s) in 1 object(s) allocated from:
1: #0 0x7fbdffab61e7 in operator new(unsigned long) ../../../../src/libsanitizer/asan/asan_new_delete.cpp:99
1: #1 0x559086696289 in redGrapes::Context::init(unsigned long, std::shared_ptr<redGrapes::scheduler::IScheduler>) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:132
1: #2 0x55908669647f in redGrapes::Context::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.cpp:140
1: #3 0x5590865d49d3 in redGrapes::init(unsigned long) /home/widera/workspace/redGrapes/redGrapes/redGrapes.hpp:122
1: #4 0x5590865c8f09 in CATCH2_INTERNAL_TEST_0 /home/widera/workspace/redGrapes/test/resource.cpp:41
1: #5 0x5590867a04da in Catch::TestInvokerAsFunction::invoke() const src/catch2/internal/catch_test_case_registry_impl.cpp:149
1: #6 0x559086782f9f in Catch::TestCaseHandle::invoke() const (/home/widera/workspace/buildRedGrapes/redGrapes_test+0x22bf9f)
1: #7 0x559086780717 in Catch::RunContext::invokeActiveTestCase() src/catch2/internal/catch_run_context.cpp:538
1: #8 0x55908677fed0 in Catch::RunContext::runCurrentTest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/catch2/internal/catch_run_context.cpp:501
1: #9 0x55908677b7b0 in Catch::RunContext::runTest(Catch::TestCaseHandle const&) src/catch2/internal/catch_run_context.cpp:232
1: #10 0x5590866befbe in execute src/catch2/catch_session.cpp:110
1: #11 0x5590866c1f40 in Catch::Session::runInternal() src/catch2/catch_session.cpp:332
1: #12 0x5590866c1446 in Catch::Session::run() src/catch2/catch_session.cpp:263
1: #13 0x5590866bcfab in int Catch::Session::run<char>(int, char const* const*) src/catch2/../catch2/catch_session.hpp:41
1: #14 0x5590866bcd66 in main src/catch2/internal/catch_main.cpp:36
1: #15 0x7fbdff229d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
1:
1: SUMMARY: AddressSanitizer: 720 byte(s) leaked in 6 allocation(s).
1/1 Test #1: unittest .........................***Failed 2.61 sec
Rebased again, and I'm not able to break it so far, all tests look fine.
Only concern I have now is the use of
offsetof
inwhere I get the warning that this is 'conditionally supported' , but it seems to work fine.
If we inherit from memory::Refcounted< TaskSpace, TaskSpaceDeleter >::Guard
we should be able to static cast *this
to get access to the element.
This PR implements some memory-layout optimizations which in turn will hopefully improve cache utilization. The majority of bytes saved come from a custom implementation of reference-counting, where a smart-pointer reference only allocates a single pointer (8byte) instead of the fat 16-byte
std::shared_ptr
. Additionally the members of some structs have been reordered to reduce space wasted by alignment. Another place where some bytes are gained is to implementnull
-types directly instead of usingstd::optional
(e.g. forEventPtr
).Minimal Task-Config (only Resource-, Graph- & IdProperty)
Task
TaskProperties
Extended Task-Config (+ LabelProperty, SchedulingTag)
Task
TaskProperties
IDProperty
ResourceProperty
GraphProperty
Event
EventPtr
ChunkedList<uint8_t>
ChunkedList<uint8_t>::Item
AtomicList<uint8_t>
AtomicList<uint8_t>::ItemLink