jonstewart
commented
1 year ago I get the same, not sure what it is yet.
Closed juliapaluch closed 1 year ago
jonstewart
commented
1 year ago I get the same, not sure what it is yet.
simsong
commented
1 year ago Thanks for the report. Can you tell me what platform you are using? Specifically, which compiler and which OS?
jonstewart
commented
1 year ago I’m macOS Catalina, 10.15.7, Intel, Apple clang 12.0.0. I believe Julia has an Intel Mac, too. Let me check on my M1…
simsong
commented
1 year ago So it may be the new compiler?
simsong
commented
1 year ago Happy to report that I have replicated this on my M1 mac.
(base) simsong@Seasons bulk_extractor % src/test_be (slg-dev)bulk_extractor
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp3f9307f58827cbe" for email
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp55fc8dc0b351b8de" for email
check "/Users/simsong/gits/bulk_extractor/src/tests"
Test directory: "/Users/simsong/gits/bulk_extractor/src/tests"
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp873ebbe1ab07c542" for pdf
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp7fb7eca5b04d1067" for email
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmpd4ca5e511ebd80d5" for exif
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp107498d013e30c3a" for exif
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp3a37008cac2f82d3" for exif
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp1dd8b2bf797e8515" for json
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmpb57b7ce0365a3915" for vcard_carved
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmpaf2671a1c4a01a97" for wordlist
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp9c499a58b338df3" for winprefetch
## output in "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmp8b1a326b391ffe51" for zip
## image_fname: ram_2pages.bin outdir: "/var/folders/hp/8jsmkdrj6cnfk_0x105st5zr0000gn/T/be_tmpa761e9dd71566d8e"
=================================================================
==23770==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x00016aef1ccc at pc 0x0001050755b4 bp 0x00016aef18d0 sp 0x00016aef18c8
READ of size 4 at 0x00016aef1ccc thread T0
#0 0x1050755b0 in create_aes128_schedule(unsigned char const*, unsigned char*) scan_aes.cpp
#1 0x105368090 in validate_aes128_key(unsigned char*) test_be2.cpp:308
#2 0x1053684bc in ____C_A_T_C_H____T_E_S_T____4() test_be2.cpp:329
#3 0x10522fbfc in Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) catch.hpp:12989
#4 0x10522dc90 in Catch::RunContext::runTest(Catch::TestCase const&) catch.hpp:12750
#5 0x10523fb18 in Catch::Session::runInternal() catch.hpp:13549
#6 0x10523d470 in Catch::Session::run() catch.hpp:13505
#7 0x1052935a4 in main catch.hpp:17504
#8 0x1a983bf24 (<unknown module>)
Address 0x00016aef1ccc is located in stack of thread T0 at offset 12 in frame
#0 0x10522f7ac in Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) catch.hpp:12963
This frame has 6 object(s):
[32, 96) 'testCaseSection' (line 12965)
[128, 144) 'ref.tmp.sroa.5' (line 12970)
[160, 296) 'redirectedStreams' (line 12978)
[368, 370) 'dummyReaction' (line 12998)
[384, 408) 'ref.tmp38' (line 12999)
[448, 560) 'testCaseSectionStats' (line 13010)
HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
(longjmp and C++ exceptions *are* supported)
SUMMARY: AddressSanitizer: stack-buffer-underflow scan_aes.cpp in create_aes128_schedule(unsigned char const*, unsigned char*)
Shadow bytes around the buggy address:
0x00702d5fe340: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x00702d5fe350: 00 00 f2 f2 f2 f2 f2 f2 f2 f2 00 00 00 00 00 f3
0x00702d5fe360: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
0x00702d5fe370: 00 00 00 00 00 00 00 00 f1 f1 f1 f1 00 00 f2 f2
0x00702d5fe380: 00 00 f2 f2 00 00 f3 f3 00 00 00 00 00 00 00 00
=>0x00702d5fe390: 00 00 00 00 00 00 00 00 f1[f1]f1 f1 00 00 00 00
0x00702d5fe3a0: 00 00 00 00 f2 f2 f2 f2 f8 f8 f2 f2 f8 f8 f8 f8
0x00702d5fe3b0: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f2 f2 f2
0x00702d5fe3c0: f2 f2 f2 f2 f2 f2 f8 f2 f8 f8 f8 f2 f2 f2 f2 f2
0x00702d5fe3d0: f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f8 f3 f3
0x00702d5fe3e0: f3 f3 f3 f3 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==23770==ABORTING
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_be is a Catch v2.13.6 host application.
Run with -? for options
-----------------------------------------------------------------------------------------------------------------------
schedule_aes
-----------------------------------------------------------------------------------------------------------------------
test_be2.cpp:323
.......................................................................................................................
test_be2.cpp:323: FAILED:
due to a fatal error condition:
SIGABRT - Abort (abnormal termination) signal
=======================================================================================================================
test cases: 23 | 22 passed | 1 failed
assertions: 207 | 206 passed | 1 failed
zsh: abort src/test_be
(base) simsong@Seasons bulk_extractor % (slg-dev)bulk_extractorI should have this fixed by Sunday PM.
jonstewart
commented
1 year ago Repro’d on my M1, too. Do you mind giving us a week? This looks fun to debug. :-D
simsong
commented
1 year ago (lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = Stack buffer underflow
* frame #0: 0x000000010135cd2c libclang_rt.asan_osx_dynamic.dylib`__asan::AsanDie()
frame #1: 0x0000000101376c38 libclang_rt.asan_osx_dynamic.dylib`__sanitizer::Die() + 192
frame #2: 0x000000010135ac2c libclang_rt.asan_osx_dynamic.dylib`__asan::ScopedInErrorReport::~ScopedInErrorReport() + 1124
frame #3: 0x0000000101359f04 libclang_rt.asan_osx_dynamic.dylib`__asan::ReportGenericError(unsigned long, unsigned long, unsigned long, unsigned long, bool, unsigned long, unsigned int, bool) + 1460
frame #4: 0x000000010135b9b4 libclang_rt.asan_osx_dynamic.dylib`__asan_report_load_n + 56
frame #5: 0x00000001001695b4 test_be`create_aes128_schedule(key=<unavailable>, computed="") at scan_aes.cpp:0:1 [opt]
frame #6: 0x000000010045c094 test_be`validate_aes128_key(key="") at test_be2.cpp:308:5 [opt]
frame #7: 0x000000010045c4c0 test_be`____C_A_T_C_H____T_E_S_T____4() at test_be2.cpp:329:5 [opt]
frame #8: 0x0000000100323c00 test_be`Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) [inlined] Catch::TestCase::invoke(this=<unavailable>) const at catch.hpp:14156:15 [opt]
frame #9: 0x0000000100323bc8 test_be`Catch::RunContext::runCurrentTest(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >&) [inlined] Catch::RunContext::invokeActiveTestCase(this=0x000000016fdfe660) at catch.hpp:13016:27 [opt]
frame #10: 0x0000000100323bac test_be`Catch::RunContext::runCurrentTest(this=0x000000016fdfe660, redirectedCout=<unavailable>, redirectedCerr=<unavailable>) at catch.hpp:12989:17 [opt]
frame #11: 0x0000000100321c94 test_be`Catch::RunContext::runTest(this=0x000000016fdfe660, testCase=<unavailable>) at catch.hpp:12750:13 [opt]
frame #12: 0x0000000100333b1c test_be`Catch::Session::runInternal() at catch.hpp:13343:45 [opt]
frame #13: 0x0000000100333794 test_be`Catch::Session::runInternal(this=<unavailable>) at catch.hpp:13549:39 [opt]
frame #14: 0x0000000100331474 test_be`Catch::Session::run(this=0x000000016fdfeca0) at catch.hpp:13505:24 [opt]
frame #15: 0x00000001003875a8 test_be`main [inlined] int Catch::Session::run<char>(this=0x000000016fdfeca0, argc=2, argv=<unavailable>) at catch.hpp:13227:30 [opt]
frame #16: 0x0000000100387540 test_be`main(argc=<unavailable>, argv=<unavailable>) at catch.hpp:17504:29 [opt]
frame #17: 0x00000001a983bf28 dyld`start + 2236
(lldb)Repro’d on my M1, too. Do you mind giving us a week? This looks fun to debug. :-D
I was really planning on working out tonight. If you want, you can have until 8am tomorrow.
simsong
commented
1 year ago Okay, I found the bug. Let me know if/when you want me to push.
jonstewart
commented
1 year ago Nuts!On Apr 28, 2023, at 9:26 PM, Simson L. Garfinkel @.***> wrote: Okay, I found the bug. Let me know if/when you want me to push.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>
simsong
commented
1 year ago I take it that's a yes/no/maybe?
sf-jonstewart
commented
1 year ago Yup, that looks like the right fix.
I was looking forward to accepting the challenge, right after I finished watching TV, and you just went ahead and fixed it... Nuts!
simsong
commented
1 year ago If you are looking for something fun, please try one of these:
Say the word and I will happily assign them to you.
jonstewart
commented
1 year ago We're working on scan_lightgrep and stumbled across this AES issue in so doing.
Contents of
src/test-suite.log:Output of
git show:Output of
./bootstrap.sh && LDFLAGS="-fsanitize=address -fno-omit-frame-pointer" CXXFLAGS="-fsanitize=address -fno-omit-frame-pointer -g -Og" ./configure:Output of
make -j8 check: