Open Muqi-Zou opened 4 years ago
We recently developed an auto testing tool, which supports the address sanitizer (Asan), for the sgx application. For the testing of Cxx11SGXDemo, we added -fsanitize=address into the Enclave_C_Flags, -lasan -lm into Enclave_Link_Flags. And we found three unknown Asan crashes in the libcxx/include:
- The error is reported as:
==21995==ERROR: AddressSanitizer: unknown-crash on address 0x7ffff4f9f0d0 at pc 0x7ffff4131f39 bp 0x7ffff5509330 sp 0x7ffff5509320 WRITE of size 8 at 0x7ffff4f9f0d0 thread T0
when the unordered_set constructs itself:
std::unordered_set<int> set_of_numbers = { 0, 1, 2, 3, 4, 5 };
fromvoid ecall_new_container_classes_demo()
in Enclave/TrustedLibrary/Libcxx.cpp To be more specific: The _Nodeptr__next_
could not be accessed when running__h->__next_ = nullptr;
insidetypename __hash_table<_Tp, _Hash, _Equal, _Alloc>::__node_holder __hash_table<_Tp, _Hash, _Equal, _Alloc>::__construct_node_hash( size_t __hash, _First&& __f, _Rest&& ...__rest) { __new_node->__left_ = nullptr; __new_node->__right_ = nullptr; __new_node->__parent_ = __parent; // __new_node->__is_black_ is initialized in __tree_balance_after_insert __child = __new_node; if (__begin_node()->__left_ != nullptr) __begin_node() = static_cast<__iter_pointer>(__begin_node()->__left_); __tree_balance_after_insert(__end_node()->__left_, __child); ++size(); }
in libcxx/__hash_table
- The error is reported as:
==17585==ERROR: AddressSanitizer: unknown-crash on address 0x7ffff4f9f0f8 at pc 0x7ffff413cdd7 bp 0x7ffff55099e0 sp 0x7ffff55099d0 WRITE of size 8 at 0x7ffff4f9f0f8 thread T0
when the map<char,char> constructs itself:
std::map<char, char> map_of_letters = { {'B','b' } };
fromvoid ecall_tuple_demo()
in Enclave/TrustedLibrary/Libcxx.cpp To be more specific: The pointer__right_
could not be accessed when running__new_node->__right_ = nullptr;
insidevoid __tree<_Tp, _Compare, _Allocator>::__insert_node_at( __parent_pointer __parent, __node_base_pointer& __child, __node_base_pointer __new_node){ static_assert(!__is_hash_value_type<_First, _Rest...>::value, "Construct cannot be called with a hash value type"); __node_allocator& __na = __node_alloc(); __node_holder __h(__node_traits::allocate(__na, 1), _Dp(__na)); __node_traits::construct(__na, _NodeTypes::__get_ptr(__h->__value_), _VSTD::forward<_First>(__f), _VSTD::forward<_Rest>(__rest)...); __h.get_deleter().__value_constructed = true; __h->__hash_ = __hash; __h->__next_ = nullptr; return __h; }
in libcxx/__tree
- The error is reported as:
==11433==ERROR: AddressSanitizer: unknown-crash on address 0x7ffff4f9f0d0 at pc 0x7ffff412d223 bp 0x7ffff550a7c0 sp 0x7ffff550a7b0 WRITE of size 8 at 0x7ffff4f9f0d0 thread T0
when the shared_ptr constructs itself:
auto shared_ptr = std::make_shared<DemoSmartPtr>("smart_ptr.");
fromvoid ecall_shared_ptr_demo()
in Enclave/TrustedLibrary/Libcxx.cpp To be more specific: The function _shared_count(long __refs = 0)
could not be accessed inclass _LIBCPP_TYPE_VIS __shared_count { __shared_count(const __shared_count&); __shared_count& operator=(const __shared_count&); protected: long __shared_owners_; virtual ~__shared_count(); private: virtual void __on_zero_shared() _NOEXCEPT = 0; public: _LIBCPP_INLINE_VISIBILITY explicit __shared_count(long __refs = 0) _NOEXCEPT : __shared_owners_(__refs) {}
For the debugging, here is what we have tried:
- We tried to run the same code using the libc++ without the sgx-sdk, there is not crash reported by the sanitizer. And we believed, with high probability, it is not a false positive relating with the sanitizer.
- We checked the process of the ./app running, the malloc (sbrk) being called during the allocation is inside the tlibc/stdlib/malloc.c (tlibc/gen/sbrk.c).
- Since some of the codes of libcxx have not been updated for few years, we compared two files, map and __tree, in sgxsdk/include/llibcxx with the llvm's (https://github.com/llvm/llvm-project/tree/master/libcxx/include). We patched them with the current version of llvm's besides the function/class relating with _NodeTypes and unique_ptr and the crash still exists.
We suspect that something inside the unique_ptr causes the crash, since the make_shared also refers the unique_ptr. However, we have limited knowledge about the how unique_ptr and shared_ptr are constructed. We hope the information provided could help you find some potential bugs in this project.
For the detailed back trace info: 1.
Breakpoint 1, libsfuzz_ocall_isatty (retval=0x7ffff5507fd8, fd=2) at Enclave/Enclave_t.c:834 834 sgx_status_t status = SGX_SUCCESS; #0 libsfuzz_ocall_isatty (retval=0x7ffff5507fd8, fd=2) at Enclave/Enclave_t.c:834 #1 0x00007ffff4149d68 in isatty (fd=2) at libsgx_tsgxfuzz.c:782 #2 0x00007ffff4294e19 in __sanitizer::SupportsColoredOutput(int) () #3 0x00007ffff42a15af in __sanitizer::ReportFile::SupportsColors() () #4 0x00007ffff42bbdcc in __asan::ErrorGeneric::Print() () #5 0x00007ffff428cf60 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) () #6 0x00007ffff428e28a in __asan_report_store8 () #7 0x00007ffff4131f39 in std::__1::__hash_table<int, std::__1::hash<int>, std::__1::equal_to<int>, std::__1::allocator<int> >::__construct_node_hash<int const&> (this=0x7ffff550b450, __hash=0, __f=@0x7ffff42d7d80: 0) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/__hash_table:2284 #8 0x00007ffff4122557 in std::__1::__hash_table<int, std::__1::hash<int>, std::__1::equal_to<int>, std::__1::allocator<int> >::__emplace_unique_key_args<int, int const&> (__k=@0x7ffff42d7d80: 0, this=0x7ffff550b450) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/__hash_table:2000 #9 std::__1::__hash_table<int, std::__1::hash<int>, std::__1::equal_to<int>, std::__1::allocator<int> >::__insert_unique (__x=@0x7ffff42d7d80: 0, this=0x7ffff550b450) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/__hash_table:1140 #10 std::__1::unordered_set<int, std::__1::hash<int>, std::__1::equal_to<int>, std::__1::allocator<int> >::insert<int const*> (__last=0x7ffff42d7d98, __first=0x7ffff42d7d80 <._38>, this=0x7ffff550b450) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/unordered_set:833 #11 std::__1::unordered_set<int, std::__1::hash<int>, std::__1::equal_to<int>, std::__1::allocator<int> >::unordered_set (this=0x7ffff550b450, __il=...) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/unordered_set:766 #12 0x00007ffff410d85e in ecall_new_container_classes_demo () at Enclave/TrustedLibrary/Libcxx.cpp:594
Breakpoint 1, libsfuzz_ocall_isatty (retval=0x7ffff5508218, fd=2) at Enclave/Enclave_t.c:834 834 sgx_status_t status = SGX_SUCCESS; #0 libsfuzz_ocall_isatty (retval=0x7ffff5508218, fd=2) at Enclave/Enclave_t.c:834 #1 0x00007ffff4149d68 in isatty (fd=2) at libsgx_tsgxfuzz.c:782 #2 0x00007ffff4294e19 in __sanitizer::SupportsColoredOutput(int) () #3 0x00007ffff42a15af in __sanitizer::ReportFile::SupportsColors() () #4 0x00007ffff42bbdcc in __asan::ErrorGeneric::Print() () #5 0x00007ffff428cf60 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) () #6 0x00007ffff428e28a in __asan_report_store8 () #7 0x00007ffff413cdd7 in std::__1::__tree<std::__1::__value_type<char, char>, std::__1::__map_value_compare<char, std::__1::__value_type<char, char>, std::__1::less<char>, true>, std::__1::allocator<std::__1::__value_type<char, char> > >::__insert_node_at (this=0x7ffff550b550, __parent=0x7ffff550b558, __child=@0x7ffff550b558: 0x0, __new_node=0x7ffff4f9f0f0) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/__tree:2058 #8 0x00007ffff413823a in std::__1::__tree<std::__1::__value_type<char, char>, std::__1::__map_value_compare<char, std::__1::__value_type<char, char>, std::__1::less<char>, true>, std::__1::allocator<std::__1::__value_type<char, char> > >::__emplace_hint_unique_key_args<char, std::__1::pair<char const, char> const&> (this=0x7ffff550b550, __p=..., __k=@0x7ffff550a790: 66 'B') at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/__tree:2124 #9 0x00007ffff4111c9f in std::__1::__tree<std::__1::__value_type<char, char>, std::__1::__map_value_compare<char, std::__1::__value_type<char, char>, std::__1::less<char>, true>, std::__1::allocator<std::__1::__value_type<char, char> > >::__insert_unique (__v=..., __p=..., this=0x7ffff550b550) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/__tree:1242 #10 std::__1::map<char, char, std::__1::less<char>, std::__1::allocator<std::__1::pair<char const, char> > >::insert (__v=..., __p=..., this=0x7ffff550b550) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/map:1126 #11 std::__1::map<char, char, std::__1::less<char>, std::__1::allocator<std::__1::pair<char const, char> > >::insert<std::__1::pair<char const, char> const*> (__l=0x7ffff550a792, __f=0x7ffff550a790, this=0x7ffff550b550) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/map:1147 #12 std::__1::map<char, char, std::__1::less<char>, std::__1::allocator<std::__1::pair<char const, char> > >::map (__comp=..., __il=..., this=<optimized out>) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/map:1002 #13 ecall_tuple_demo () at Enclave/TrustedLibrary/Libcxx.cpp:638
Breakpoint 1, libsfuzz_ocall_isatty (retval=0x7ffff5509078, fd=2) at Enclave/Enclave_t.c:834 834 sgx_status_t status = SGX_SUCCESS; #0 libsfuzz_ocall_isatty (retval=0x7ffff5509078, fd=2) at Enclave/Enclave_t.c:834 #1 0x00007ffff4149d68 in isatty (fd=2) at libsgx_tsgxfuzz.c:782 #2 0x00007ffff4294e19 in __sanitizer::SupportsColoredOutput(int) () #3 0x00007ffff42a15af in __sanitizer::ReportFile::SupportsColors() () #4 0x00007ffff42bbdcc in __asan::ErrorGeneric::Print() () #5 0x00007ffff428cf60 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) () #6 0x00007ffff428e28a in __asan_report_store8 () #7 0x00007ffff412d223 in std::__1::__shared_count::__shared_count (__refs=0, this=0x7ffff4f9f0d0) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/memory:3722 #8 std::__1::__shared_weak_count::__shared_weak_count (__refs=0, this=0x7ffff4f9f0d0) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/memory:3741 #9 std::__1::__shared_ptr_emplace<DemoSmartPtr, std::__1::allocator<DemoSmartPtr> >::__shared_ptr_emplace<char const (&) [11]> (__a=..., this=0x7ffff4f9f0d0) at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/memory:3832 #10 std::__1::shared_ptr<DemoSmartPtr>::make_shared<char const (&) [11]> () at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/memory:4447 #11 0x00007ffff41146d7 in std::__1::make_shared<DemoSmartPtr, char const (&) [11]> () at /mnt/data/muqi/patchedSDK/sgxsdk//include/libcxx/memory:4811 #12 ecall_shared_ptr_demo () at Enclave/TrustedLibrary/Libcxx.cpp:691 #13 0x00007ffff41025b9 in sgx_ecall_shared_ptr_demo (pms=0x0) at Enclave/Enclave_t.c:419
Hello, I have encountered a similar problem to yours. I want to use addresssanitizer in the enclave code, but an error has been reported when compiling. Have you encountered a similar situation? I am very grateful for your help
Enclave.cpp:(.text._sub_D_00099_0+0x15): undefined reference to `__asan_unregister_globals'
/usr/local/bin/ld: Enclave/Enclave.o: in function `_sub_I_00099_1':
Enclave.cpp:(.text._sub_I_00099_1+0x9): undefined reference to `__asan_init'
/usr/local/bin/ld: Enclave.cpp:(.text._sub_I_00099_1+0xe): undefined reference to `__asan_version_mismatch_check_v8'
/usr/local/bin/ld: Enclave.cpp:(.text._sub_I_00099_1+0x1f): undefined reference to `__asan_register_globals'
/usr/local/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status
make: *** [Makefile:241:enclave.so] 错误 1
We recently developed an auto testing tool, which supports the address sanitizer (Asan), for the sgx application. For the testing of Cxx11SGXDemo, we added -fsanitize=address into the Enclave_C_Flags, -lasan -lm into Enclave_Link_Flags. And we found three unknown Asan crashes in the libcxx/include:
The error is reported as:
when the unordered_set constructs itself:
std::unordered_set<int> set_of_numbers = { 0, 1, 2, 3, 4, 5 };
fromvoid ecall_new_container_classes_demo()
in Enclave/TrustedLibrary/Libcxx.cpp To be more specific: The _Nodeptr__next_
could not be accessed when running__h->__next_ = nullptr;
insidein libcxx/__hash_table
The error is reported as:
when the map<char,char> constructs itself:
std::map<char, char> map_of_letters = { {'B','b' } };
fromvoid ecall_tuple_demo()
in Enclave/TrustedLibrary/Libcxx.cpp To be more specific: The pointer__right_
could not be accessed when running__new_node->__right_ = nullptr;
insidein libcxx/__tree
The error is reported as:
when the shared_ptr constructs itself:
auto shared_ptr = std::make_shared<DemoSmartPtr>("smart_ptr.");
fromvoid ecall_shared_ptr_demo()
in Enclave/TrustedLibrary/Libcxx.cpp To be more specific: The function _shared_count(long __refs = 0)
could not be accessed inprotected: long sharedowners; virtual ~shared_count(); private: virtual void __on_zero_shared() _NOEXCEPT = 0;
public: _LIBCPP_INLINE_VISIBILITY explicit shared_count(long __refs = 0) _NOEXCEPT : sharedowners(__refs) {}
Breakpoint 1, libsfuzz_ocall_isatty (retval=0x7ffff5507fd8, fd=2) at Enclave/Enclave_t.c:834 834 sgx_status_t status = SGX_SUCCESS;
0 libsfuzz_ocall_isatty (retval=0x7ffff5507fd8, fd=2) at Enclave/Enclave_t.c:834
1 0x00007ffff4149d68 in isatty (fd=2) at libsgx_tsgxfuzz.c:782
2 0x00007ffff4294e19 in __sanitizer::SupportsColoredOutput(int) ()
3 0x00007ffff42a15af in __sanitizer::ReportFile::SupportsColors() ()
4 0x00007ffff42bbdcc in __asan::ErrorGeneric::Print() ()
5 0x00007ffff428cf60 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) ()
6 0x00007ffff428e28a in __asan_report_store8 ()
7 0x00007ffff4131f39 in std::1::hash_table<int, std::1::hash, std:: 1::equal_to, std::1::allocator >:: construct_node_hash<int const&> (this=0x7ffff550b450, __hash=0,
8 0x00007ffff4122557 in std::1::hash_table<int, std::1::hash, std:: 1::equal_to, std::1::allocator >:: emplace_unique_key_args<int, int const&> (__k=@0x7ffff42d7d80: 0,
9 std::1::hash_table<int, std::1::hash, std:: 1::equal_to, std::1::allocator >:: insert_unique (__x=@0x7ffff42d7d80: 0, this=0x7ffff550b450)
10 std::1::unordered_set<int, std::1::hash, std:: 1::equal_to, std:: 1::allocator >::insert<int const*> (last=0x7ffff42d7d98, first=0x7ffff42d7d80 <._38>, this=0x7ffff550b450)
11 std::1::unordered_set<int, std::1::hash, std:: 1::equal_to, std:: 1::allocator >::unordered_set (this=0x7ffff550b450, __il=...)
12 0x00007ffff410d85e in ecall_new_container_classes_demo ()
Breakpoint 1, libsfuzz_ocall_isatty (retval=0x7ffff5508218, fd=2) at Enclave/Enclave_t.c:834 834 sgx_status_t status = SGX_SUCCESS;
0 libsfuzz_ocall_isatty (retval=0x7ffff5508218, fd=2) at Enclave/Enclave_t.c:834
1 0x00007ffff4149d68 in isatty (fd=2) at libsgx_tsgxfuzz.c:782
2 0x00007ffff4294e19 in __sanitizer::SupportsColoredOutput(int) ()
3 0x00007ffff42a15af in __sanitizer::ReportFile::SupportsColors() ()
4 0x00007ffff42bbdcc in __asan::ErrorGeneric::Print() ()
5 0x00007ffff428cf60 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) ()
6 0x00007ffff428e28a in __asan_report_store8 ()
7 0x00007ffff413cdd7 in std::1::tree<std::1::value_type<char, char>, std::1::map_value_compare<char, std::1::value_type<char, char>, std::1::less, true>, std:: 1::allocator<std::1::value_type<char, char> > >::__insert_node_at (this=0x7ffff550b550,
8 0x00007ffff413823a in std::1::tree<std::1::value_type<char, char>, std::1::map_value_compare<char, std::1::value_type<char, char>, std::1::less, true>, std:: 1::allocator<std::1::value_type<char, char> > >::emplace_hint_unique_key_args<char, std::1::pair<char const, char> const&> (this=0x7ffff550b550, p=..., k=@0x7ffff550a790: 66 'B')
9 0x00007ffff4111c9f in std::1::tree<std::1::value_type<char, char>, std::1::map_value_compare<char, std::1::value_type<char, char>, std::1::less, true>, std:: 1::allocator<std::1::value_type<char, char> > >::__insert_unique (v=..., p=..., this=0x7ffff550b550)
10 std::1::map<char, char, std::1::less, std::1::allocator<std::1::pair<char const, char> > >::insert (v=..., p=..., this=0x7ffff550b550)
11 std::1::map<char, char, std::1::less, std::1::allocator<std::1::pair<char const, char> > >::insert<std::1::pair<char const, char> const*> (l=0x7ffff550a792, __f=0x7ffff550a790,
12 std::1::map<char, char, std::1::less, std::1::allocator<std::1::pair<char const, char> > >::map (comp=..., il=..., this=)
13 ecall_tuple_demo () at Enclave/TrustedLibrary/Libcxx.cpp:638
Breakpoint 1, libsfuzz_ocall_isatty (retval=0x7ffff5509078, fd=2) at Enclave/Enclave_t.c:834 834 sgx_status_t status = SGX_SUCCESS;
0 libsfuzz_ocall_isatty (retval=0x7ffff5509078, fd=2) at Enclave/Enclave_t.c:834
1 0x00007ffff4149d68 in isatty (fd=2) at libsgx_tsgxfuzz.c:782
2 0x00007ffff4294e19 in __sanitizer::SupportsColoredOutput(int) ()
3 0x00007ffff42a15af in __sanitizer::ReportFile::SupportsColors() ()
4 0x00007ffff42bbdcc in __asan::ErrorGeneric::Print() ()
5 0x00007ffff428cf60 in __asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) ()
6 0x00007ffff428e28a in __asan_report_store8 ()
7 0x00007ffff412d223 in std::1::shared_count::__shared_count (__refs=0, this=0x7ffff4f9f0d0)
8 std::1::shared_weak_count::shared_weak_count (refs=0, this=0x7ffff4f9f0d0)
9 std::1::shared_ptr_emplace<DemoSmartPtr, std::1::allocator >:: shared_ptr_emplace<char const (&) [11]> (__a=..., this=0x7ffff4f9f0d0)
10 std::__1::shared_ptr::make_shared<char const (&) [11]> ()
11 0x00007ffff41146d7 in std::__1::make_shared<DemoSmartPtr, char const (&) [11]> ()
12 ecall_shared_ptr_demo () at Enclave/TrustedLibrary/Libcxx.cpp:691
13 0x00007ffff41025b9 in sgx_ecall_shared_ptr_demo (pms=0x0) at Enclave/Enclave_t.c:419