search

awslabs / aws-crt-cpp

C++ wrapper around the aws-c-* libraries. Provides Cross-Platform Transport Protocols and SSL/TLS implementations for C++.
Apache License 2.0
73 stars 64 forks source link

Fix ASan report about alloc/dealloc mismatch when compiled with C++23 #629

Closed alexey-milovidov closed 2 months ago

alexey-milovidov commented 2 months ago

Description of changes:

When the library is compiled with -std=c++23, ASan starts to complain about mismatched allocation/deallocation, like the following:

2024-06-16 08:26:29 =================================================================
2024-06-16 08:26:29 ==2168==ERROR: AddressSanitizer: alloc-dealloc-mismatch (operator new vs free) on 0x506000109040
2024-06-16 08:26:29     #0 0x558a1c76a236 in free (/usr/bin/clickhouse+0xa4fb236) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #1 0x558a45097fd4 in std::__1::vector<aws_host_address, Aws::Crt::StlAllocator<aws_host_address>>::push_back[abi:v15000](aws_host_address const&) HostResolver.cpp
2024-06-16 08:26:29     #2 0x558a45097a14 in Aws::Crt::Io::DefaultHostResolver::s_onHostResolved(aws_host_resolver*, aws_string const*, int, aws_array_list const*, void*) (/usr/bin/clickhouse+0x32e28a14) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #3 0x558a45092762 in aws_host_resolver_thread host_resolver.c
2024-06-16 08:26:29     #4 0x558a44fe282c in thread_fn thread.c
2024-06-16 08:26:29     #5 0x558a1c768058 in asan_thread_start(void*) crtstuff.c
2024-06-16 08:26:29     #6 0x7f719ff95ac2  (/lib/x86_64-linux-gnu/libc.so.6+0x94ac2) (BuildId: 962015aa9d133c6cbcfb31ec300596d7f44d3348)
2024-06-16 08:26:29     #7 0x7f71a002784f  (/lib/x86_64-linux-gnu/libc.so.6+0x12684f) (BuildId: 962015aa9d133c6cbcfb31ec300596d7f44d3348)
2024-06-16 08:26:29 
2024-06-16 08:26:29 0x506000109040 is located 0 bytes inside of 64-byte region [0x506000109040,0x506000109080)
2024-06-16 08:26:29 allocated by thread T3 here:
2024-06-16 08:26:29     #0 0x558a1c79ddbd in operator new(unsigned long) (/usr/bin/clickhouse+0xa52edbd) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #1 0x558a45097f1f in std::__1::vector<aws_host_address, Aws::Crt::StlAllocator<aws_host_address>>::push_back[abi:v15000](aws_host_address const&) HostResolver.cpp
2024-06-16 08:26:29     #2 0x558a45097a14 in Aws::Crt::Io::DefaultHostResolver::s_onHostResolved(aws_host_resolver*, aws_string const*, int, aws_array_list const*, void*) (/usr/bin/clickhouse+0x32e28a14) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #3 0x558a45092762 in aws_host_resolver_thread host_resolver.c
2024-06-16 08:26:29     #4 0x558a44fe282c in thread_fn thread.c
2024-06-16 08:26:29     #5 0x558a1c768058 in asan_thread_start(void*) crtstuff.c
2024-06-16 08:26:29 
2024-06-16 08:26:29 Thread T3 created by T0 here:
2024-06-16 08:26:29     #0 0x558a1c74ff91 in pthread_create (/usr/bin/clickhouse+0xa4e0f91) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #1 0x558a44fe22fb in aws_thread_launch (/usr/bin/clickhouse+0x32d732fb) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #2 0x558a4508e64f in default_resolve_host host_resolver.c
2024-06-16 08:26:29     #3 0x558a450983f3 in Aws::Crt::Io::DefaultHostResolver::ResolveHost(std::__1::basic_string<char, std::__1::char_traits<char>, Aws::Crt::StlAllocator<char>> const&, std::__1::function<void (Aws::Crt::Io::HostResolver&, std::__1::vector<aws_host_address, Aws::Crt::StlAllocator<aws_host_address>> const&, int)> const&) (/usr/bin/clickhouse+0x32e293f3) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #4 0x558a44acfd4a in Aws::Auth::GeneralHTTPCredentialsProvider::ShouldCreateGeneralHTTPProvider(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>) (/usr/bin/clickhouse+0x32860d4a) (BuildId: 9a4ef85e15c9099607afed77eac4c9d0390ac9f2)
2024-06-16 08:26:29     #5 0x558a37df6c01 in decltype(std::declval<bool (*&)(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>)>()(std::declval<std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>

The reason is that StlAllocator is derived from std::allocator, but in C++23, std::allocator has a new method allocate_at_least, which is defined by default to call allocate. As allocate is not a virtual method, it ends up calling std::allocator<T>::allocate which calls operator new. At the same time, deallocate is called from StlAllocator, which calls free.

I've added the missing method. Also, added a call to reserve while investigating this issue.

This issue was found by the ClickHouse continuous integration system, here: https://github.com/ClickHouse/ClickHouse/pull/65164

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

waahm7 commented 2 months ago

Thank you for creating the PR. LGTM but there are some trivial lint issues. I have created another PR with the fixes at #633.