Open n-bes opened 1 month ago
I tried to reproduce the issue on Fedora 40 x86-64, but GCC fails with:
# gcc hello.c -g -fsanitize=hwaddress -o hello && ./hello
cc1: warning: '-fsanitize=hwaddress' is not supported for this target
FATAL: HWAddressSanitizer requires a kernel with tagged address ABI.
export CC=/usr/bin/gcc
I don't understand if you try to build Python with clang or gcc. Your build script says GCC, but logs mention Clang:
See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags
faulthandler.c uses a weak heuristic to compute the stack size, see gh-65330:
/* bpo-21131: allocate dedicated stack of SIGSTKSZ*2 bytes, instead of just
SIGSTKSZ bytes. Calling the previous signal handler in faulthandler
signal handler uses more than SIGSTKSZ bytes of stack memory on some
platforms. */
stack.ss_size = SIGSTKSZ * 2;
I tried to reproduce the issue on Fedora 40 x86-64
HWasan supported only on aarch64
Your build script says GCC, but logs mention Clang
Hm... Wrong isolation? Later, I'll try to remove clang toolchain and re-rerun build.
Hm... Wrong isolation? Later, I'll try to remove clang toolchain and re-rerun build.
Maybe it's just GCC which points to clang documentation.
To check if the stack size is the issue, you can modify faulthandler.c to replace stack.ss_size = SIGSTKSZ * 2;
with stack.ss_size = SIGSTKSZ * 10;
. Do you still have the issue?
Do you still have the issue?
Yes
Changes:
FROM ubuntu:24.10
RUN apt-get update -y && \
apt-get install -y \
autoconf \
build-essential \
curl \
gcc \
gdb \
gnupg \
libbz2-dev \
liblzma-dev \
libncurses5-dev \
libncursesw5-dev \
libreadline-dev \
libsqlite3-dev \
libssl-dev \
lld \
lsb-release \
make \
pkg-config \
rsync \
software-properties-common \
tk-dev \
wget \
wget \
xz-utils \
zlib1g-dev
WORKDIR /src/
COPY Python-3.13.0 Python-3.13.0
COPY build.sh .
#!/bin/bash
set -x
set -e
# print OS version
cat /etc/os-release
# print current env
env
export CC=/usr/bin/gcc
export CXX=/usr/bin/g++
export CFLAGS="-g -fsanitize=hwaddress"
export CCFLAGS="-g -fsanitize=hwaddress"
export CXXFLAGS="-g -fsanitize=hwaddress"
export CPPFLAGS="-g -fsanitize=hwaddress"
export LDFLAGS="-fsanitize=hwaddress"
export HWASAN_OPTIONS="detect_leaks=0:allocator_may_return_null=1:handle_segv=0"
cd /src/Python-3.13.0
set +e
if ! ./configure --with-assertions --with-pydebug; then
cat config.log
exit 1
else
make
./python -m test -uall
fi
AssertionError: b'==10440==ERROR: HWAddressSanitizer: tag-mismatch on address 0xeeadffe20020 at pc 0xffff96c28374
WRITE of size 25208 at 0xeeadffe20020 tags: b8/00 (ptr/mem) in thread T0
Invalid access starting at offset 20352
#0 0xffff96c28374 in SigTrap<(__hwasan::ErrorAction)1, (__hwasan::AccessType)1> ../../../../src/libsanitizer/hwasan/hwasan_checks.h:107
#1 0xffff96c28374 in CheckAddressSized<(__hwasan::ErrorAction)1, (__hwasan::AccessType)1> ../../../../src/libsanitizer/hwasan/hwasan_checks.h:178
#2 0xffff96c28374 in memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:87
#3 0xffff96c28374 in memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:85
#4 0xaaaae0936b7c in memset /usr/include/aarch64-linux-gnu/bits/string_fortified.h:59
#5 0xaaaae0936b7c in fill_mem_debug Objects/obmalloc.c:2632
#6 0xaaaae0938a08 in _PyMem_DebugRawFree Objects/obmalloc.c:2761
#7 0xaaaae0938b28 in _PyMem_DebugFree Objects/obmalloc.c:2899
#8 0xaaaae0957464 in PyMem_Free Objects/obmalloc.c:1018
#9 0xaaaae0b63734 in _PyFaulthandler_Fini Modules/faulthandler.c:1409
#10 0xaaaae0b19c4c in _Py_Finalize Python/pylifecycle.c:2123
#11 0xaaaae0b1a438 in Py_Exit Python/pylifecycle.c:3392
#12 0xaaaae0b23514 in handle_system_exit Python/pythonrun.c:635
#13 0xaaaae0b239ac in _PyErr_PrintEx Python/pythonrun.c:644
#14 0xaaaae0b23d80 in PyErr_PrintEx Python/pythonrun.c:721
#15 0xaaaae0b23d98 in PyErr_Print Python/pythonrun.c:727
#16 0xaaaae0b2478c in _PyRun_SimpleStringFlagsWithName Python/pythonrun.c:552
#17 0xaaaae0b5cd68 in pymain_run_command Modules/main.c:253
#18 0xaaaae0b5e0bc in pymain_run_python Modules/main.c:687
#19 0xaaaae0b5e6e0 in Py_RunMain Modules/main.c:775
#20 0xaaaae0b5e81c in pymain_main Modules/main.c:805
#21 0xaaaae0b5ea58 in Py_BytesMain Modules/main.c:829
#22 0xaaaae07d3e70 in main Programs/python.c:15
#23 0xffff969b2290 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
#24 0xffff969b2374 in __libc_start_main_impl ../csu/libc-start.c:360
#25 0xaaaae07d3d6c in _start (/src/Python-3.13.0/python+0x83d6c) (BuildId: 68636a20b8aec9aecfcfaa930dd35b1a859630f3)
[0xeeadffe20000,0xeeadffe27000) is a small allocated heap chunk; size: 28672 offset: 32
Cause: heap-buffer-overflow
0xeeadffe20020 is located 32 bytes inside a 25248-byte region [0xeeadffe20000,0xeeadffe262a0)
allocated by thread T0 here:
#0 0xffff96c16848 in __sanitizer_malloc ../../../../src/libsanitizer/hwasan/hwasan_allocation_functions.cpp:151
#1 0xaaaae0937ba8 in _PyMem_RawMalloc Objects/obmalloc.c:62
#2 0xaaaae0936c84 in _PyMem_DebugRawAlloc Objects/obmalloc.c:2694
#3 0xaaaae0936cd4 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2727
#4 0xaaaae0956674 in PyMem_RawMalloc Objects/obmalloc.c:948
#5 0xaaaae0958f7c in _PyObject_Malloc Objects/obmalloc.c:2140
#6 0xaaaae0936c84 in _PyMem_DebugRawAlloc Objects/obmalloc.c:2694
#7 0xaaaae0936cd4 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2727
#8 0xaaaae0938aa0 in _PyMem_DebugMalloc Objects/obmalloc.c:2884
#9 0xaaaae09573b4 in PyMem_Malloc Objects/obmalloc.c:989
#10 0xaaaae0b61ba4 in faulthandler_allocate_stack Modules/faulthandler.c:417
#11 0xaaaae0b61ef4 in faulthandler_enable Modules/faulthandler.c:447
#12 0xaaaae0b62844 in faulthandler_py_enable Modules/faulthandler.c:519
#13 0xaaaae092ad98 in cfunction_call Objects/methodobject.c:540
#14 0xaaaae08af294 in _PyObject_MakeTpCall Objects/call.c:242
#15 0xaaaae08af510 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:166
#16 0xaaaae08af540 in PyObject_CallNoArgs Objects/call.c:106
#17 0xaaaae0b63458 in faulthandler_init_enable Modules/faulthandler.c:1322
#18 0xaaaae0b63578 in _PyFaulthandler_Init Modules/faulthandler.c:1357
#19 0xaaaae0b170b8 in init_interp_main Python/pylifecycle.c:1194
Thread: T0 0xeffe00002000 stack: [0xffffeab7e000,0xffffeb37e000) sz: 8388608 tls: [0xffff9729e660,0xffff9729f520)
Memory tags around the buggy address (one tag corresponds to 16 bytes):
0xeeadffe24700: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe24800: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe24900: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe24a00: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe24b00: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe24c00: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe24d00: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe24e00: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
=>0xeeadffe24f00: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 [00] 00 b8 b8 b8 b8
0xeeadffe25000: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe25100: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe25200: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe25300: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe25400: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe25500: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe25600: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
0xeeadffe25700: b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8 b8
Tags for short granules around the buggy address (one tag corresponds to 16 bytes):
0xeeadffe24e00: .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
=>0xeeadffe24f00: .. .. .. .. .. .. .. .. .. .. [..] .. .. .. .. ..
0xeeadffe25000: .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags
SUMMARY: HWAddressSanitizer: tag-mismatch ../../../../src/libsanitizer/hwasan/hwasan_checks.h:107 in SigTrap<(__hwasan::ErrorAction)1, (__hwasan::AccessType)1>
' is not false
$ sed -n 1342p Python-3.13.0/Modules/faulthandler.c
stack.ss_size = SIGSTKSZ * 10;
AssertionError: b'==10440==ERROR: HWAddressSanitizer: tag-mismatch on address 0xeeadffe20020 at pc 0xffff93228374
WRITE of size 25208 at 0xeeadffe20020 tags: 56/00 (ptr/mem) in thread T0
Invalid access starting at offset 20352
#0 0xffff93228374 in SigTrap<(__hwasan::ErrorAction)1, (__hwasan::AccessType)1> ../../../../src/libsanitizer/hwasan/hwasan_checks.h:107
#1 0xffff93228374 in CheckAddressSized<(__hwasan::ErrorAction)1, (__hwasan::AccessType)1> ../../../../src/libsanitizer/hwasan/hwasan_checks.h:178
#2 0xffff93228374 in memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:87
#3 0xffff93228374 in memset ../../../../src/libsanitizer/sanitizer_common/sanitizer_common_interceptors_memintrinsics.inc:85
#4 0xaaaac7876b7c in memset /usr/include/aarch64-linux-gnu/bits/string_fortified.h:59
#5 0xaaaac7876b7c in fill_mem_debug Objects/obmalloc.c:2632
#6 0xaaaac7878a08 in _PyMem_DebugRawFree Objects/obmalloc.c:2761
#7 0xaaaac7878b28 in _PyMem_DebugFree Objects/obmalloc.c:2899
#8 0xaaaac7897464 in PyMem_Free Objects/obmalloc.c:1018
#9 0xaaaac7aa3738 in _PyFaulthandler_Fini Modules/faulthandler.c:1409
#10 0xaaaac7a59c4c in _Py_Finalize Python/pylifecycle.c:2123
#11 0xaaaac7a5a438 in Py_Exit Python/pylifecycle.c:3392
#12 0xaaaac7a63514 in handle_system_exit Python/pythonrun.c:635
#13 0xaaaac7a639ac in _PyErr_PrintEx Python/pythonrun.c:644
#14 0xaaaac7a63d80 in PyErr_PrintEx Python/pythonrun.c:721
#15 0xaaaac7a63d98 in PyErr_Print Python/pythonrun.c:727
#16 0xaaaac7a6478c in _PyRun_SimpleStringFlagsWithName Python/pythonrun.c:552
#17 0xaaaac7a9cd68 in pymain_run_command Modules/main.c:253
#18 0xaaaac7a9e0bc in pymain_run_python Modules/main.c:687
#19 0xaaaac7a9e6e0 in Py_RunMain Modules/main.c:775
#20 0xaaaac7a9e81c in pymain_main Modules/main.c:805
#21 0xaaaac7a9ea58 in Py_BytesMain Modules/main.c:829
#22 0xaaaac7713e70 in main Programs/python.c:15
#23 0xffff93062290 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
#24 0xffff93062374 in __libc_start_main_impl ../csu/libc-start.c:360
#25 0xaaaac7713d6c in _start (/src/Python-3.13.0/python+0x83d6c) (BuildId: 7c4c3c0f8c673b2e452296c82d12be22033b969a)
[0xeeadffe20000,0xeeadffe27000) is a small allocated heap chunk; size: 28672 offset: 32
Cause: heap-buffer-overflow
0xeeadffe20020 is located 32 bytes inside a 25248-byte region [0xeeadffe20000,0xeeadffe262a0)
allocated by thread T0 here:
#0 0xffff93216848 in __sanitizer_malloc ../../../../src/libsanitizer/hwasan/hwasan_allocation_functions.cpp:151
#1 0xaaaac7877ba8 in _PyMem_RawMalloc Objects/obmalloc.c:62
#2 0xaaaac7876c84 in _PyMem_DebugRawAlloc Objects/obmalloc.c:2694
#3 0xaaaac7876cd4 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2727
#4 0xaaaac7896674 in PyMem_RawMalloc Objects/obmalloc.c:948
#5 0xaaaac7898f7c in _PyObject_Malloc Objects/obmalloc.c:2140
#6 0xaaaac7876c84 in _PyMem_DebugRawAlloc Objects/obmalloc.c:2694
#7 0xaaaac7876cd4 in _PyMem_DebugRawMalloc Objects/obmalloc.c:2727
#8 0xaaaac7878aa0 in _PyMem_DebugMalloc Objects/obmalloc.c:2884
#9 0xaaaac78973b4 in PyMem_Malloc Objects/obmalloc.c:989
#10 0xaaaac7aa1ba4 in faulthandler_allocate_stack Modules/faulthandler.c:417
#11 0xaaaac7aa1ef4 in faulthandler_enable Modules/faulthandler.c:447
#12 0xaaaac7aa2844 in faulthandler_py_enable Modules/faulthandler.c:519
#13 0xaaaac786ad98 in cfunction_call Objects/methodobject.c:540
#14 0xaaaac77ef294 in _PyObject_MakeTpCall Objects/call.c:242
#15 0xaaaac77ef510 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:166
#16 0xaaaac77ef540 in PyObject_CallNoArgs Objects/call.c:106
#17 0xaaaac7aa3458 in faulthandler_init_enable Modules/faulthandler.c:1322
#18 0xaaaac7aa357c in _PyFaulthandler_Init Modules/faulthandler.c:1357
#19 0xaaaac7a570b8 in init_interp_main Python/pylifecycle.c:1194
Thread: T0 0xeffe00002000 stack: [0xffffca363000,0xffffcab63000) sz: 8388608 tls: [0xffff939ed660,0xffff939ee520)
Memory tags around the buggy address (one tag corresponds to 16 bytes):
0xeeadffe24700: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe24800: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe24900: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe24a00: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe24b00: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe24c00: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe24d00: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe24e00: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
=>0xeeadffe24f00: 56 56 56 56 56 56 56 56 56 56 [00] 00 56 56 56 56
0xeeadffe25000: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe25100: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe25200: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe25300: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe25400: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe25500: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe25600: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
0xeeadffe25700: 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56 56
Tags for short granules around the buggy address (one tag corresponds to 16 bytes):
0xeeadffe24e00: .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
=>0xeeadffe24f00: .. .. .. .. .. .. .. .. .. .. [..] .. .. .. .. ..
0xeeadffe25000: .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. ..
See https://clang.llvm.org/docs/HardwareAssistedAddressSanitizerDesign.html#short-granules for a description of short granule tags
SUMMARY: HWAddressSanitizer: tag-mismatch ../../../../src/libsanitizer/hwasan/hwasan_checks.h:107 in SigTrap<(__hwasan::ErrorAction)1, (__hwasan::AccessType)1>
' is not false
Bug report
Bug description:
Enviroment:
Build script:
Log (full):
CPython versions tested on:
3.13
Operating systems tested on:
Linux