search

libgit2 / pygit2

Python bindings for libgit2
https://www.pygit2.org/
Other
1.58k stars 382 forks source link

Segmentation fault at `test_push_options()` #1301

Closed anatol closed 2 weeks ago

anatol commented 3 weeks ago

I am porting my company system to the latest libgit2/pygit2 and I stuck due to a test error, a memory violation to be specific.

It happens in test_push_options() at this exact line

assert ffi.string(remote_push_options.strings[0]) == b'foo'

I ran the test with a debugger and I see that remote_push_options looks fine, remote_push_options.strings is a valid object, but remote_push_options.strings[0] points to some garbage address like 0x2d2d2d2d2d2d2d. When ffi.string() tries to dereference the address it crashes.

So I ran the test with address sanitizer and I got heap-use-after-free error, i.e. address pointing by remote_push_options.strings[0] been freed somewhere else.

Here are the address stanitizer stack traces, though they are for the C stack rather than for python code:

SanitizerError
AddressSanitizer: heap-use-after-free py/cffi/c/_cffi_backend.c:1081:29 in convert_to_object
Details
=================================================================
==5627==ERROR: AddressSanitizer: heap-use-after-free on address 0x506000c28bb0 at pc 0x7f03360c6842 bp 0x7fff1d8496d0 sp 0x7fff1d8496c8
READ of size 8 at 0x506000c28bb0 thread T0
    #0 0x7f03360c6841 in convert_to_object py/cffi/c/_cffi_backend.c:1081:29
    #1 0x7f03360d5cf5 in cdata_subscript py/cffi/c/_cffi_backend.c:2719:12
    #2 0x55a35cc41b8b in PyObject_GetItem python_runtime/v3_11/Objects/abstract.c:157:26
    #3 0x55a35cdd6a36 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:2139:29
    #4 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #5 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #6 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #7 0x55a35cc6a8c2 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:257:24
    #8 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #9 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #10 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #11 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #12 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #13 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #14 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #15 0x55a35cc6a8c2 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:257:24
    #16 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #17 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #18 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #19 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #20 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #21 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #22 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #23 0x55a35cc6a8f8 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:245:16
    #24 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #25 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #26 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #27 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #28 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #29 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #30 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #31 0x55a35cc69c34 in _PyObject_FastCallDictTstate python_runtime/v3_11/Objects/call.c:152:15
    #32 0x55a35cc6b1f0 in _PyObject_Call_Prepend python_runtime/v3_11/Objects/call.c:482:24
    #33 0x55a35ccf6b36 in slot_tp_call python_runtime/v3_11/Objects/typeobject.c:7628:15
    #34 0x55a35cc69f56 in _PyObject_MakeTpCall python_runtime/v3_11/Objects/call.c:214:18
    #35 0x55a35cc6aa39 in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:90:16
    #36 0x55a35cc6a992 in PyObject_Vectorcall python_runtime/v3_11/Objects/call.c:299:12
    #37 0x55a35cdd7644 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c
    #38 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #39 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #40 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #41 0x55a35cc6a8f8 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:245:16
    #42 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #43 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #44 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #45 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #46 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #47 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #48 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #49 0x55a35cc69c34 in _PyObject_FastCallDictTstate python_runtime/v3_11/Objects/call.c:152:15
    #50 0x55a35cc6b1f0 in _PyObject_Call_Prepend python_runtime/v3_11/Objects/call.c:482:24
    #51 0x55a35ccf6b36 in slot_tp_call python_runtime/v3_11/Objects/typeobject.c:7628:15
    #52 0x55a35cc6ac48 in _PyObject_Call python_runtime/v3_11/Objects/call.c:343:18
    #53 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #54 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #55 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #56 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #57 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #58 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #59 0x55a35cc6a8f8 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:245:16
    #60 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #61 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #62 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #63 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #64 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #65 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #66 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #67 0x55a35cc6a8f8 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:245:16
    #68 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #69 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #70 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #71 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #72 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #73 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #74 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #75 0x55a35cc69c34 in _PyObject_FastCallDictTstate python_runtime/v3_11/Objects/call.c:152:15
    #76 0x55a35cc6b1f0 in _PyObject_Call_Prepend python_runtime/v3_11/Objects/call.c:482:24
    #77 0x55a35ccf6b36 in slot_tp_call python_runtime/v3_11/Objects/typeobject.c:7628:15
    #78 0x55a35cc69f56 in _PyObject_MakeTpCall python_runtime/v3_11/Objects/call.c:214:18
    #79 0x55a35cc6aa39 in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:90:16
    #80 0x55a35cc6a992 in PyObject_Vectorcall python_runtime/v3_11/Objects/call.c:299:12
    #81 0x55a35cdd7644 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c
    #82 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #83 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #84 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #85 0x55a35cc6a8f8 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:245:16
    #86 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #87 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #88 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #89 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #90 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #91 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #92 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #93 0x55a35cc69c34 in _PyObject_FastCallDictTstate python_runtime/v3_11/Objects/call.c:152:15
    #94 0x55a35cc6b1f0 in _PyObject_Call_Prepend python_runtime/v3_11/Objects/call.c:482:24
    #95 0x55a35ccf6b36 in slot_tp_call python_runtime/v3_11/Objects/typeobject.c:7628:15
    #96 0x55a35cc69f56 in _PyObject_MakeTpCall python_runtime/v3_11/Objects/call.c:214:18
    #97 0x55a35cc6aa39 in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:90:16
    #98 0x55a35cc6a992 in PyObject_Vectorcall python_runtime/v3_11/Objects/call.c:299:12
    #99 0x55a35cdd7644 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c
    #100 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #101 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #102 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #103 0x55a35cc6a8f8 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:245:16
    #104 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #105 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #106 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #107 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #108 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #109 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #110 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #111 0x55a35cc69c34 in _PyObject_FastCallDictTstate python_runtime/v3_11/Objects/call.c:152:15
    #112 0x55a35cc6b1f0 in _PyObject_Call_Prepend python_runtime/v3_11/Objects/call.c:482:24
    #113 0x55a35ccf6b36 in slot_tp_call python_runtime/v3_11/Objects/typeobject.c:7628:15
    #114 0x55a35cc69f56 in _PyObject_MakeTpCall python_runtime/v3_11/Objects/call.c:214:18
    #115 0x55a35cc6aa39 in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:90:16
    #116 0x55a35cc6a992 in PyObject_Vectorcall python_runtime/v3_11/Objects/call.c:299:12
    #117 0x55a35cdd7644 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c
    #118 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #119 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #120 0x55a35cdcbaac in PyEval_EvalCode python_runtime/v3_11/Python/ceval.c:1149:21
    #121 0x55a35cdc7055 in builtin_exec_impl python_runtime/v3_11/Python/bltinmodule.c:1077:17
    #122 0x55a35cdc7055 in builtin_exec python_runtime/v3_11/Python/clinic/bltinmodule.c.h:465:20
    #123 0x55a35ccc998c in cfunction_vectorcall_FASTCALL_KEYWORDS python_runtime/v3_11/Objects/methodobject.c:443:24
    #124 0x55a35cc6aa11 in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:92:11
    #125 0x55a35cc6a992 in PyObject_Vectorcall python_runtime/v3_11/Objects/call.c:299:12
    #126 0x55a35cdd7644 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c
    #127 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #128 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #129 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #130 0x55a35cc6b48d in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:92:11
    #131 0x55a35cc6b48d in _PyObject_CallFunctionVa python_runtime/v3_11/Objects/call.c
    #132 0x55a35cc6b7c1 in callmethod python_runtime/v3_11/Objects/call.c:608:12
    #133 0x55a35cc6b7c1 in PyObject_CallMethod python_runtime/v3_11/Objects/call.c:627:24
    #134 0x55a35c6a9d9c in python_launcher::RunFileAsMain(char const*, _object*) devtools/python/launcher/launcher_common.cc:114:12
    #135 0x55a35c6a9a31 in RunMainFromImporter devtools/python/launcher/launcher_common.cc:385:11
    #136 0x55a35c6a9a31 in python_launcher::Launcher_Main(int, char**, int (*)(int*, char**), int (*)(int*, char**), int (*)(char const*), bool) devtools/python/launcher/launcher_common.cc:655:14
    #137 0x55a35c6a8330 in python_launcher::HermeticLauncherMain(int, char**, bool) devtools/python/launcher/hermetic_launcher_lib.cc:116:10
    #138 0x55a35c49b7fa in main devtools/python/launcher/hermetic_launcher.cc:4:10
    #139 0x7f033b64e3d3 in __libc_start_main (/usr/lib64/libc.so.6+0x613d3) (BuildId: 9a996398ce14a94560b0c642eb4f6e94)
    #140 0x55a35c3c48e9 in _start /usr/debug-src/src/csu/../sysdeps/x86_64/start.S:120

0x506000c28bb0 is located 48 bytes inside of 56-byte region [0x506000c28b80,0x506000c28bb8)
freed by thread T0 here:
    #0 0x55a35c461a26 in free llvm/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3
    #1 0x7f03360d7ae0 in cdata_dealloc py/cffi/c/_cffi_backend.c:1893:5
    #2 0x7f03360d7ae0 in cdataowning_dealloc py/cffi/c/_cffi_backend.c:1918:5
    #3 0x55a35ccd1a94 in _Py_Dealloc python_runtime/v3_11/Objects/object.c:2390:5
    #4 0x55a35cc87d92 in Py_DECREF python_runtime/v3_11/Include/object.h:538:9
    #5 0x55a35cc87d92 in Py_XDECREF python_runtime/v3_11/Include/object.h:602:9
    #6 0x55a35cc87d92 in _PyObject_FreeInstanceAttributes python_runtime/v3_11/Objects/dictobject.c:5583:9
    #7 0x55a35cce83f9 in subtype_dealloc python_runtime/v3_11/Objects/typeobject.c:1433:13
    #8 0x55a35ccd1a94 in _Py_Dealloc python_runtime/v3_11/Objects/object.c:2390:5
    #9 0x55a35ce12a79 in Py_DECREF python_runtime/v3_11/Include/object.h:538:9
    #10 0x55a35ce12a79 in Py_XDECREF python_runtime/v3_11/Include/object.h:602:9
    #11 0x55a35ce12a79 in _PyFrame_Clear python_runtime/v3_11/Python/frame.c:141:9
    #12 0x55a35cde186f in _PyEvalFrameClearAndPop python_runtime/v3_11/Python/ceval.c:6414:5
    #13 0x55a35cddb82d in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:2443:17
    #14 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #15 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #16 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #17 0x55a35cc6a8c2 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:257:24
    #18 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #19 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #20 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #21 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #22 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #23 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #24 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #25 0x55a35cc6a8c2 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:257:24
    #26 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #27 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #28 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #29 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #30 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #31 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #32 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #33 0x55a35cc6a8f8 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:245:16
    #34 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #35 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #36 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #37 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #38 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #39 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #40 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #41 0x55a35cc69c34 in _PyObject_FastCallDictTstate python_runtime/v3_11/Objects/call.c:152:15

previously allocated by thread T0 here:
    #0 0x55a35c461ea9 in calloc llvm/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:75:3
    #1 0x7f03360d70da in allocate_owning_object py/cffi/c/_cffi_backend.c:3747:14
    #2 0x7f03360e93c2 in allocate_with_allocator py/cffi/c/_cffi_backend.c:3813:14
    #3 0x7f03360e8f62 in direct_newp py/cffi/c/_cffi_backend.c:3946:14
    #4 0x7f03360ddfd6 in _ffi_new py/cffi/c/ffi_obj.c:349:12
    #5 0x7f03360ddfd6 in ffi_new py/cffi/c/ffi_obj.c:354:12
    #6 0x55a35ccca6cd in cfunction_call python_runtime/v3_11/Objects/methodobject.c:542:18
    #7 0x55a35cc69f56 in _PyObject_MakeTpCall python_runtime/v3_11/Objects/call.c:214:18
    #8 0x55a35cc6aa39 in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:90:16
    #9 0x55a35cc6a992 in PyObject_Vectorcall python_runtime/v3_11/Objects/call.c:299:12
    #10 0x55a35cdd7644 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c
    #11 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #12 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #13 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #14 0x55a35cc69c6b in _PyObject_FastCallDictTstate python_runtime/v3_11/Objects/call.c:141:15
    #15 0x55a35cc6b1f0 in _PyObject_Call_Prepend python_runtime/v3_11/Objects/call.c:482:24
    #16 0x55a35ccf7926 in slot_tp_init python_runtime/v3_11/Objects/typeobject.c:7859:15
    #17 0x55a35ccec1c6 in type_call python_runtime/v3_11/Objects/typeobject.c:1103:19
    #18 0x55a35cc69f56 in _PyObject_MakeTpCall python_runtime/v3_11/Objects/call.c:214:18
    #19 0x55a35cc6aa39 in _PyObject_VectorcallTstate python_runtime/v3_11/Include/internal/pycore_call.h:90:16
    #20 0x55a35cc6a992 in PyObject_Vectorcall python_runtime/v3_11/Objects/call.c:299:12
    #21 0x55a35cdd7644 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c
    #22 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #23 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #24 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #25 0x55a35cc6a8c2 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:257:24
    #26 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16
    #27 0x55a35cc6adaf in PyObject_Call python_runtime/v3_11/Objects/call.c:355:12
    #28 0x55a35cdd8c55 in do_call_core python_runtime/v3_11/Python/ceval.c
    #29 0x55a35cdd8c55 in _PyEval_EvalFrameDefault python_runtime/v3_11/Python/ceval.c:5384:22
    #30 0x55a35cdcbc0f in _PyEval_EvalFrame python_runtime/v3_11/Include/internal/pycore_ceval.h:94:16
    #31 0x55a35cdcbc0f in _PyEval_Vector python_runtime/v3_11/Python/ceval.c:6442:24
    #32 0x55a35cc6af05 in _PyFunction_Vectorcall python_runtime/v3_11/Objects/call.c
    #33 0x55a35cc6a8c2 in _PyVectorcall_Call python_runtime/v3_11/Objects/call.c:257:24
    #34 0x55a35cc6ac01 in _PyObject_Call python_runtime/v3_11/Objects/call.c:328:16

SUMMARY: AddressSanitizer: heap-use-after-free py/cffi/c/_cffi_backend.c:1081:29 in convert_to_object
anatol commented 3 weeks ago

I am looking at this code https://github.com/libgit2/pygit2/blob/a9c7cce9eaff27b5c65bd23a35406b314c16922b/pygit2/utils.py#L112 and the comment says:

The above construct is still subject to FFI scoping rules, i.e. the
 contents of 'struct' only remain valid within the StrArray context.

which makes me believe that in this block https://github.com/libgit2/pygit2/blob/a9c7cce9eaff27b5c65bd23a35406b314c16922b/pygit2/remotes.py#L270 pushopts will be freed when the function is exited. Thus the test_push_options uses invalid memory.

jdavid commented 3 weeks ago

ping @kempniu

@anatol With what OS and CPU arch do you see this issue?

jdavid commented 3 weeks ago

For context push options were implemented in PR #1282

anatol commented 3 weeks ago

I am using Linux at x86_64, AMD hardware.

kempniu commented 3 weeks ago

Yeah, it looks like test_push_options() is faulty indeed. I feared that while implementing it originally (see https://github.com/libgit2/pygit2/pull/1282#issuecomment-2018194432) and it seems we were just relying on favorable memory allocator behavior so far. Extending CI tests for pygit2 with an ASAN-enabled libgit2/pygit2 build would be great as that would catch issues like this in time.

Like I said originally in #1282, I don't have any bright ideas for testing this because we basically want to peek into a structure that only lives ephemerally inside pygit2/remotes.py:push() and there is no infrastructure for having a server-side canary that could inspect things at the other end (server), which is what libgit2 has. Given all this, test_push_options() should probably be removed :shrug:

kempniu commented 2 weeks ago

Alternatively, to keep some form of limited testing for push options, just the assertions for string contents can be removed:

diff --git a/test/test_remote.py b/test/test_remote.py
index ee42535..dce094f 100644
--- a/test/test_remote.py
+++ b/test/test_remote.py
@@ -373,10 +373,7 @@ def test_push_options(mock_callbacks, origin, clone, remote):
     remote.push(['refs/heads/master'], push_options=['foo'])
     remote_push_options = mock_callbacks.return_value.push_options.remote_push_options
     assert remote_push_options.count == 1
-    assert ffi.string(remote_push_options.strings[0]) == b'foo'

     remote.push(['refs/heads/master'], push_options=['Option A', 'Option B'])
     remote_push_options = mock_callbacks.return_value.push_options.remote_push_options
     assert remote_push_options.count == 2
-    assert ffi.string(remote_push_options.strings[0]) == b'Option A'
-    assert ffi.string(remote_push_options.strings[1]) == b'Option B'

This is safe to do because the git_push_options structure contains the whole git_strarray remote_push_options; structure, not just a pointer to it, so the count member of remote_push_options will remain within valid memory even outside of the StrArray context manager.

jdavid commented 2 weeks ago

Cool, would you do a PR?

kempniu commented 2 weeks ago

Sure, see #1304.