search

skupperproject / skupper-router

An application-layer router for Skupper networks
https://skupper.io
Apache License 2.0
14 stars 18 forks source link

AddressSanitizer: heap-use-after-free from Proton proactor in post_event ../c/src/proactor/epoll.c:2303 #1400

Open jiridanek opened 7 months ago

jiridanek commented 7 months ago

https://github.com/skupperproject/skupper-router/actions/runs/7798062635/job/21266090300#step:10:4048

 ==7399==ERROR: AddressSanitizer: heap-use-after-free on address 0x61b000176684 at pc 0x7fef6213c6cc bp 0x7fef56dcd040 sp 0x7fef56dcd030
62: READ of size 4 at 0x61b000176684 thread T5
62:     #0 0x7fef6213c6cb in post_event ../c/src/proactor/epoll.c:2303
62:     #1 0x7fef621413c8 in poller_do_epoll ../c/src/proactor/epoll.c:2602
62:     #2 0x7fef6213f8bf in next_event_batch ../c/src/proactor/epoll.c:2486
62:     #3 0x7fef62144b5f in pn_proactor_wait ../c/src/proactor/epoll.c:2725
62:     #4 0x55ea885ed7ba in thread_run ../src/server.c:1144
62:     #5 0x55ea88474427 in _thread_init ../src/posix/threading.c:207
62:     #6 0x7fef61094ac2 in start_thread nptl/pthread_create.c:442
62:     #7 0x7fef6112684f  (/lib/x86_64-linux-gnu/libc.so.6+0x12684f) (BuildId: c289da5071a3399de893d2af81d6a30c62646e1e)
62: 
62: 0x61b000176684 is located 1284 bytes inside of 1536-byte region [0x61b000176180,0x61b000176780)
62: freed by thread T3 here:
62:     #0 0x7fef626ddb20 in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:52
62:     #1 0x7fef62149bce in praw_connection_cleanup ../c/src/proactor/epoll_raw_connection.c:166
62:     #2 0x7fef6214cb39 in pni_raw_connection_done ../c/src/proactor/epoll_raw_connection.c:469
62:     #3 0x7fef62144cbb in pn_proactor_done ../c/src/proactor/epoll.c:2747
62:     #4 0x55ea885edad7 in thread_run ../src/server.c:1188
62:     #5 0x55ea88474427 in _thread_init ../src/posix/threading.c:207
62:     #6 0x7fef61094ac2 in start_thread nptl/pthread_create.c:442
62: 
62: previously allocated by thread T5 here:
62:     #0 0x7fef626de997 in __interceptor_calloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:77
62:     #1 0x7fef62149bf0 in pn_raw_connection ../c/src/proactor/epoll_raw_connection.c:172
62:     #2 0x55ea8836ee9f in qdr_tcp_connection_ingress ../src/adaptors/tcp/tcp_adaptor.c:1338
62:     #3 0x55ea883a63ab in _listener_event_handler ../src/adaptors/adaptor_listener.c:153
62:     #4 0x55ea885e8b06 in handle_event_with_context ../src/server.c:813
62:     #5 0x55ea885e8b8c in do_handle_listener ../src/server.c:824
62:     #6 0x55ea885eb9d6 in handle ../src/server.c:1022
62:     #7 0x55ea885ed8c1 in thread_run ../src/server.c:1170
62:     #8 0x55ea88474427 in _thread_init ../src/posix/threading.c:207
62:     #9 0x7fef61094ac2 in start_thread nptl/pthread_create.c:442
62: 
62: Thread T5 created by T0 here:
62:     #0 0x7fef62649175 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:208
62:     #1 0x55ea884748ea in sys_thread ../src/posix/threading.c:229
62:     #2 0x55ea885f3838 in qd_server_run ../src/server.c:1539
62:     #3 0x55ea886002d1 in main_process ../router/src/main.c:111
62:     #4 0x55ea88602446 in main ../router/src/main.c:365
62:     #5 0x7fef61029d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
62: 
62: Thread T3 created by T0 here:
62:     #0 0x7fef62649175 in __interceptor_pthread_create ../../../../src/libsanitizer/asan/asan_interceptors.cpp:208
62:     #1 0x55ea884748ea in sys_thread ../src/posix/threading.c:229
62:     #2 0x55ea885f3838 in qd_server_run ../src/server.c:1539
62:     #3 0x55ea886002d1 in main_process ../router/src/main.c:111
62:     #4 0x55ea88602446 in main ../router/src/main.c:365
62:     #5 0x7fef61029d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
62: 
62: SUMMARY: AddressSanitizer: heap-use-after-free ../c/src/proactor/epoll.c:2303 in post_event
62: Shadow bytes around the buggy address:
62:   0x61b000176400: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
62:   0x61b000176480: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
62:   0x61b000176500: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
62:   0x61b000176580: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
62:   0x61b000176600: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
62: =>0x61b000176680:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
62:   0x61b000176700: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
62:   0x61b000176780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
62:   0x61b000176800: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
62:   0x61b000176880: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
62:   0x61b000176900: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
62: Shadow byte legend (one shadow byte represents 8 application bytes):
62:   Addressable:           00
62:   Partially addressable: 01 02 03 04 05 06 07 
62:   Heap left redzone:       fa
62:   Freed heap region:       fd
62:   Stack left redzone:      f1
62:   Stack mid redzone:       f2
62:   Stack right redzone:     f3
62:   Stack after return:      f5
62:   Stack use after scope:   f8
62:   Global redzone:          f9
62:   Global init order:       f6
62:   Poisoned by user:        f7
62:   Container overflow:      fc
62:   Array cookie:            ac
62:   Intra object redzone:    bb
62:   ASan internal:           fe
62:   Left alloca redzone:     ca
62:   Right alloca redzone:    cb
62: ==7399==ABORTING
kgiusti commented 1 month ago

https://github.com/skupperproject/skupper-router/actions/runs/9946092800/job/27475958607#step:10:3676

proton main git ref '0084354ba917f712ce293e06df5d300f9094d86e'

60: WARNING: ThreadSanitizer: heap-use-after-free (pid=4887)
60:   Write of size 1 at 0x726800051c91 by thread T5 (mutexes: write M0, write M1):
60:     #0 post_event ../c/src/proactor/epoll.c:2349 (libqpid-proton-proactor.so.1+0x137f8) (BuildId: 158eaa565e8d209417b7751d724f3f73f8099121)
60:     #1 poller_do_epoll ../c/src/proactor/epoll.c:2617 (libqpid-proton-proactor.so.1+0x137f8)
60:     #2 next_event_batch ../c/src/proactor/epoll.c:2501 (libqpid-proton-proactor.so.1+0x137f8)
60:     #3 pn_proactor_wait ../c/src/proactor/epoll.c:2740 (libqpid-proton-proactor.so.1+0x16265) (BuildId: 158eaa565e8d209417b7751d724f3f73f8099121)
60:     #4 proactor_thread ../src/server.c:168 (skrouterd+0x1317a1) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #5 _thread_init ../src/posix/threading.c:207 (skrouterd+0xc9671) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60: 
60:   Previous write of size 8 at 0x726800051c90 by thread T3:
60:     #0 free ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:724 (libtsan.so.2+0x5747c) (BuildId: 64c1e8de04b11a7d960abd7e45f94f3b277b7779)
60:     #1 praw_connection_cleanup ../c/src/proactor/epoll_raw_connection.c:171 (libqpid-proton-proactor.so.1+0x8de4) (BuildId: 158eaa565e8d209417b7751d724f3f73f8099121)
60:     #2 praw_connection_cleanup ../c/src/proactor/epoll_raw_connection.c:157 (libqpid-proton-proactor.so.1+0x8de4)
60:     #3 pni_raw_connection_done ../c/src/proactor/epoll_raw_connection.c:496 (libqpid-proton-proactor.so.1+0x174b9) (BuildId: 158eaa565e8d209417b7751d724f3f73f8099121)
60:     #4 pn_proactor_done ../c/src/proactor/epoll.c:2762 (libqpid-proton-proactor.so.1+0x174b9)
60:     #5 proactor_thread ../src/server.c:200 (skrouterd+0x131858) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #6 _thread_init ../src/posix/threading.c:207 (skrouterd+0xc9671) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60: 
60:   Mutex M0 (0x726400030f50) created at:
60:     #0 pthread_mutex_init ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1315 (libtsan.so.2+0x58bfd) (BuildId: 64c1e8de04b11a7d960abd7e45f94f3b277b7779)
60:     #1 pmutex_init ../c/src/proactor/epoll-internal.h:336 (libqpid-proton-proactor.so.1+0xcfb3) (BuildId: 158eaa565e8d209417b7751d724f3f73f8099121)
60:     #2 pn_proactor ../c/src/proactor/epoll.c:1991 (libqpid-proton-proactor.so.1+0xcfb3)
60:     #3 qd_server ../src/server.c:219 (skrouterd+0x13db59) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #4 qd_dispatch_prepare ../src/dispatch.c:343 (skrouterd+0xb239d) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #5 <null> <null> (libffi.so.8+0x7b15) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
60:     #6 main_process ../router/src/main.c:101 (skrouterd+0x13e99c) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #7 main ../router/src/main.c:366 (skrouterd+0x33fb3) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60: 
60:   Mutex M1 (0x726400030ec8) created at:
60:     #0 pthread_mutex_init ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1315 (libtsan.so.2+0x58bfd) (BuildId: 64c1e8de04b11a7d960abd7e45f94f3b277b7779)
60:     #1 pmutex_init ../c/src/proactor/epoll-internal.h:336 (libqpid-proton-proactor.so.1+0xcf86) (BuildId: 158eaa565e8d209417b7751d724f3f73f8099121)
60:     #2 pn_proactor ../c/src/proactor/epoll.c:1990 (libqpid-proton-proactor.so.1+0xcf86)
60:     #3 qd_server ../src/server.c:219 (skrouterd+0x13db59) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #4 qd_dispatch_prepare ../src/dispatch.c:343 (skrouterd+0xb239d) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #5 <null> <null> (libffi.so.8+0x7b15) (BuildId: c9149b6e99105aa4321ddd4a10ee4b90de7b7d49)
60:     #6 main_process ../router/src/main.c:101 (skrouterd+0x13e99c) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #7 main ../router/src/main.c:366 (skrouterd+0x33fb3) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60: 
60:   Thread T5 'wrkr_2' (tid=4914, running) created by main thread at:
60:     #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1022 (libtsan.so.2+0x5a267) (BuildId: 64c1e8de04b11a7d960abd7e45f94f3b277b7779)
60:     #1 sys_thread ../src/posix/threading.c:229 (skrouterd+0xcac99) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #2 qd_server_run ../src/server.c:298 (skrouterd+0x13df53) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #3 main_process ../router/src/main.c:111 (skrouterd+0x13e9f0) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #4 main ../router/src/main.c:366 (skrouterd+0x33fb3) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60: 
60:   Thread T3 'wrkr_0' (tid=4912, running) created by main thread at:
60:     #0 pthread_create ../../../../src/libsanitizer/tsan/tsan_interceptors_posix.cpp:1022 (libtsan.so.2+0x5a267) (BuildId: 64c1e8de04b11a7d960abd7e45f94f3b277b7779)
60:     #1 sys_thread ../src/posix/threading.c:229 (skrouterd+0xcac99) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #2 qd_server_run ../src/server.c:298 (skrouterd+0x13df53) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #3 main_process ../router/src/main.c:111 (skrouterd+0x13e9f0) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60:     #4 main ../router/src/main.c:366 (skrouterd+0x33fb3) (BuildId: fa4c1475441295548e7cd3bb5195382f9686ce83)
60: 
60: SUMMARY: ThreadSanitizer: heap-use-after-free ../c/src/proactor/epoll.c:2349 in post_event
kgiusti commented 1 month ago

Again but on Fedora 40:

https://github.com/skupperproject/skupper-router/actions/runs/9946092800/job/27475824204#step:34:6308

==3590==ERROR: AddressSanitizer: heap-use-after-free on address 0x51b0000d0284 at pc 0x7ff489006912 bp 0x7ff4802d1fe0 sp 0x7ff4802d1fd8
READ of size 4 at 0x51b0000d0284 thread T3
    #0 0x7ff489006911 in post_event /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll.c:2318
    #1 0x7ff48900b5cb in poller_do_epoll /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll.c:2617
    #2 0x7ff489009ad1 in next_event_batch /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll.c:2501
    #3 0x7ff48900eccd in pn_proactor_wait /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll.c:2740
    #4 0x7491c5 in proactor_thread /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:168
    #5 0x6117cd in _thread_init /home/runner/work/skupper-router/skupper-router/skupper-router/src/posix/threading.c:207
    #6 0x7ff4896bdf95 in asan_thread_start(void*) (/lib64/libasan.so.8+0x5df95) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)
    #7 0x7ff488294506 in start_thread (/lib64/libc.so.6+0x97506) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #8 0x7ff48831840b in clone3 (/lib64/libc.so.6+0x11b40b) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)

0x51b0000d0284 is located 1284 bytes inside of 1536-byte region [0x51b0000cfd80,0x51b0000d0380)
freed by thread T4 here:
    #0 0x7ff489756638 in free.part.0 (/lib64/libasan.so.8+0xf6638) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)
    #1 0x7ff489013c74 in praw_connection_cleanup /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll_raw_connection.c:171
    #2 0x7ff4890170f1 in pni_raw_connection_done /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll_raw_connection.c:496
    #3 0x7ff48900ee0f in pn_proactor_done /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll.c:2762
    #4 0x74936a in proactor_thread /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:200
    #5 0x6117cd in _thread_init /home/runner/work/skupper-router/skupper-router/skupper-router/src/posix/threading.c:207
    #6 0x7ff4896bdf95 in asan_thread_start(void*) (/lib64/libasan.so.8+0x5df95) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)

previously allocated by thread T6 here:
    #0 0x7ff489757350 in calloc (/lib64/libasan.so.8+0xf7350) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)
    #1 0x7ff489013c92 in pn_raw_connection /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll_raw_connection.c:177
    #2 0x510eec in on_accept /home/runner/work/skupper-router/skupper-router/skupper-router/src/adaptors/tcp/tcp_adaptor.c:2059
    #3 0x5197ca in _listener_event_handler /home/runner/work/skupper-router/skupper-router/skupper-router/src/adaptors/adaptor_listener.c:153
    #4 0x748ef5 in handle_event_with_context /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:107
    #5 0x748f8d in handle_listener_event /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:124
    #6 0x7492b6 in proactor_thread /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:194
    #7 0x6117cd in _thread_init /home/runner/work/skupper-router/skupper-router/skupper-router/src/posix/threading.c:207
    #8 0x7ff4896bdf95 in asan_thread_start(void*) (/lib64/libasan.so.8+0x5df95) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)

Thread T3 created by T0 here:
    #0 0x7ff48974f871 in pthread_create (/lib64/libasan.so.8+0xef871) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)
    #1 0x611bdb in sys_thread /home/runner/work/skupper-router/skupper-router/skupper-router/src/posix/threading.c:229
    #2 0x74a84d in qd_server_run /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:298
    #3 0x42fce1 in main_process /home/runner/work/skupper-router/skupper-router/skupper-router/router/src/main.c:111
    #4 0x431b91 in main /home/runner/work/skupper-router/skupper-router/skupper-router/router/src/main.c:366
    #5 0x7ff488227087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #6 0x7ff48822714a in __libc_start_main_alias_2 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #7 0x42f9e4 in _start (/__w/skupper-router/skupper-router/skupper-router/build/router/skrouterd+0x42f9e4) (BuildId: d9532c34ad3e4123096c7b4f300b7366978676c3)

Thread T4 created by T0 here:
    #0 0x7ff48974f871 in pthread_create (/lib64/libasan.so.8+0xef871) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)
    #1 0x611bdb in sys_thread /home/runner/work/skupper-router/skupper-router/skupper-router/src/posix/threading.c:229
    #2 0x74a84d in qd_server_run /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:298
    #3 0x42fce1 in main_process /home/runner/work/skupper-router/skupper-router/skupper-router/router/src/main.c:111
    #4 0x431b91 in main /home/runner/work/skupper-router/skupper-router/skupper-router/router/src/main.c:366
    #5 0x7ff488227087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #6 0x7ff48822714a in __libc_start_main_alias_2 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #7 0x42f9e4 in _start (/__w/skupper-router/skupper-router/skupper-router/build/router/skrouterd+0x42f9e4) (BuildId: d9532c34ad3e4123096c7b4f300b7366978676c3)

Thread T6 created by T0 here:
    #0 0x7ff48974f871 in pthread_create (/lib64/libasan.so.8+0xef871) (BuildId: c1431025b5d8af781c22c9ceea71f065c547d32d)
    #1 0x611bdb in sys_thread /home/runner/work/skupper-router/skupper-router/skupper-router/src/posix/threading.c:229
    #2 0x74a84d in qd_server_run /home/runner/work/skupper-router/skupper-router/skupper-router/src/server.c:298
    #3 0x42fce1 in main_process /home/runner/work/skupper-router/skupper-router/skupper-router/router/src/main.c:111
    #4 0x431b91 in main /home/runner/work/skupper-router/skupper-router/skupper-router/router/src/main.c:366
    #5 0x7ff488227087 in __libc_start_call_main (/lib64/libc.so.6+0x2a087) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #6 0x7ff48822714a in __libc_start_main_alias_2 (/lib64/libc.so.6+0x2a14a) (BuildId: 8f53abaad945a669f2bdcd25f471d80e077568ef)
    #7 0x42f9e4 in _start (/__w/skupper-router/skupper-router/skupper-router/build/router/skrouterd+0x42f9e4) (BuildId: d9532c34ad3e4123096c7b4f300b7366978676c3)

SUMMARY: AddressSanitizer: heap-use-after-free /home/runner/work/skupper-router/skupper-router/qpid-proton/c/src/proactor/epoll.c:2318 in post_event
Shadow bytes around the buggy address:
  0x51b0000d0000: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x51b0000d0080: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x51b0000d0100: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x51b0000d0180: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x51b0000d0200: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x51b0000d0280:[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x51b0000d0300: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x51b0000d0380: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x51b0000d0400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x51b0000d0480: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x51b0000d0500: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==3590==ABORTING
<<<<