search

python / cpython

The Python programming language
https://www.python.org
Other
62.14k stars 29.85k forks source link

[marshal] diffoscope crash when loading PYC files: memory corruption #121112

Open jmroot opened 2 months ago

jmroot commented 2 months ago

Crash report

What happened?

Running diffoscope version 267 under Python 3.12.4, both installed via MacPorts on Intel macOS 14.5, segfaults when given two specific files as input.

% diffoscope tortoisehg-6.6.3_0.darwin_17.noarch.tbz2 tortoisehg-6.6.3_0.darwin_23.noarch.tbz2
/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py:124: DeprecationWarning: co_lnotab is deprecated, use co_lines instead.
[many identical deprecation warnings omitted]
zsh: segmentation fault  diffoscope tortoisehg-6.6.3_0.darwin_17.noarch.tbz2

The macOS crash report from the optimized build:

Process:               Python [57729]
Path:                  /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
Identifier:            org.python.python
Version:               3.12.4 (3.12.4)
Code Type:             X86-64 (Native)
Parent Process:        zsh [826]
Responsible:           Terminal [593]
User ID:               501

Date/Time:             2024-06-28 13:01:30.2352 +1000
OS Version:            macOS 14.5 (23F79)
Report Version:        12
Bridge OS Version:     8.5 (21P5077)
Anonymous UUID:        1535D773-66C6-A888-8912-6C090207B7D2

Time Awake Since Boot: 210000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000100000000
Exception Codes:       0x0000000000000001, 0x0000000100000000

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [57729]

VM Region Info: 0x100000000 is not in any region.  Bytes before following region: 132931584
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      107ec6000-107ec8000    [    8K] r-x/r-x SM=COW  /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python

ID   Vend/Dev
6318 73ff1002
Seconds Ago   ID   Type
   210000.0   6318 Attach

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   Python                                 0x108b4462b pymalloc_alloc + 75
1   Python                                 0x108b44e9c _PyObject_Realloc + 221
2   Python                                 0x108b82c3c resize_compact + 133
3   Python                                 0x108b6fff6 _PyUnicodeWriter_Finish + 50
4   Python                                 0x108b967e3 build_string + 805
5   Python                                 0x108b94830 do_string_format + 49
6   Python                                 0x108af8fb2 method_vectorcall_VARARGS_KEYWORDS + 160
7   Python                                 0x108be3c45 _PyEval_EvalFrameDefault + 50731
8   Python                                 0x108b056ed gen_send_ex2 + 193
9   Python                                 0x108b048a6 gen_iternext + 25
10  Python                                 0x108b141a9 list_extend + 563
11  Python                                 0x108ad36cc PySequence_Fast + 168
12  Python                                 0x108b7ccf2 PyUnicode_Join + 28
13  Python                                 0x108be55d9 _PyEval_EvalFrameDefault + 57279
14  Python                                 0x108ca1f08 starmap_next + 74
15  Python                                 0x108bd05f5 filter_next + 84
16  Python                                 0x108b141a9 list_extend + 563
17  Python                                 0x108be55d9 _PyEval_EvalFrameDefault + 57279
18  Python                                 0x108ca1f08 starmap_next + 74
19  Python                                 0x108bd05f5 filter_next + 84
20  Python                                 0x108b1417b list_extend + 517
21  Python                                 0x108af92b3 method_vectorcall_O + 98
22  Python                                 0x108be3c45 _PyEval_EvalFrameDefault + 50731
23  Python                                 0x108bd745e PyEval_EvalCode + 197
24  Python                                 0x108c3dfad run_eval_code_obj + 83
25  Python                                 0x108c3c24b run_mod + 107
26  Python                                 0x108c3b865 pyrun_file + 133
27  Python                                 0x108c3ad77 _PyRun_SimpleFileObject + 287
28  Python                                 0x108c3a9d9 _PyRun_AnyFileObject + 66
29  Python                                 0x108c5f468 pymain_run_file_obj + 187
30  Python                                 0x108c5f21f pymain_run_file + 89
31  Python                                 0x108c5e773 Py_RunMain + 919
32  Python                                 0x108c5ecf8 pymain_main + 378
33  Python                                 0x108c5edab Py_BytesMain + 42
34  dyld                                0x7ff80ff51366 start + 1942

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000100000000  rbx: 0x0000000000000044  rcx: 0x000000010bc5c000  rdx: 0x000000010bd38000
  rdi: 0x0000000108f2db88  rsi: 0x0000000000000045  rbp: 0x00007ff7b8038230  rsp: 0x00007ff7b80381f0
   r8: 0x00007ff7b80383c8   r9: 0x00007ff7b8038410  r10: 0x00007ff7b8038408  r11: 0x0000000000023dfb
  r12: 0x0000000000000008  r13: 0x0000000000000004  r14: 0x0000000108f2db88  r15: 0x0000000000000045
  rip: 0x0000000108b4462b  rfl: 0x0000000000010203  cr2: 0x0000000100000000

Logical CPU:     8
Error Code:      0x00000004 (no mapping for user data read)
Trap Number:     14

Thread 0 instruction stream:
  48 03 0d de 5b 27 00 49-39 ce 49 89 ce 75 c9 eb  H...['.I9.I..u..
  95 31 c0 eb a5 55 48 89-e5 41 57 41 56 41 55 41  .1...UH..AWAVAUA
  54 53 48 83 ec 18 48 8d-86 ff fd ff ff 48 3d 00  TSH...H......H=.
  fe ff ff 0f 82 ef 03 00-00 48 89 f3 49 89 fe ff  .........H..I...
  cb 41 89 dd 41 c1 ed 04-47 8d 64 2d 00 4a 8b 0c  .A..A...G.d-.J..
  e7 48 8b 51 10 48 39 d1-74 25 ff 01 48 8b 41 08  .H.Q.H9.t%..H.A.
 [48]8b 30 48 89 71 08 48-85 f6 0f 84 c7 00 00 00  H.0H.q.H........ <==
  48 83 c4 18 5b 41 5c 41-5d 41 5e 41 5f 5d c3 4d  H...[A\A]A^A_].M
  8b be 18 02 00 00 4d 85-ff 0f 84 f6 00 00 00 49  ......M........I
  8d 57 10 41 8b 47 10 48-89 c6 4d 39 bc c6 20 02  .W.A.G.H..M9.. .
  00 00 75 0c 49 c7 84 f6-20 02 00 00 00 00 00 00  ..u.I... .......
  83 f8 02 72 0b 8d 48 ff-4d 89 bc ce 20 02 00 00  ...r..H.M... ...

Binary Images:
       0x109257000 -        0x109261fff _elementtree.cpython-312-darwin.so (*) <520c1b88-efaf-3363-a4f7-1a0fbd6dbaff> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_elementtree.cpython-312-darwin.so
       0x10923d000 -        0x10924afff _datetime.cpython-312-darwin.so (*) <327bf3d6-5bd1-339a-8157-53e63cfa20d2> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_datetime.cpython-312-darwin.so
       0x1089a0000 -        0x1089a7fff pyexpat.cpython-312-darwin.so (*) <b87f814e-5174-3b49-a31c-25b9cbfe04b1> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/pyexpat.cpython-312-darwin.so
       0x109215000 -        0x109230fff libexpat.1.9.2.dylib (*) <f4891366-5776-3908-8ebb-4725998434c3> /opt/local/lib/libexpat.1.9.2.dylib
       0x1083eb000 -        0x1083ebfff _uuid.cpython-312-darwin.so (*) <98d34c79-d6f8-306d-8664-b4d1111265ed> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_uuid.cpython-312-darwin.so
       0x1093e7000 -        0x109472fff libarchive.13.dylib (*) <88517cd2-ef65-3864-8983-930d441f34b0> /opt/local/lib/libarchive.13.dylib
       0x1089e9000 -        0x108a04fff liblzo2.2.dylib (*) <305437be-f124-365c-9d7c-e9d236bd10a6> /opt/local/lib/liblzo2.2.dylib
       0x108a60000 -        0x108a7ffff liblz4.1.9.4.dylib (*) <28e8eabe-f9ef-3635-9796-8551550c48f9> /opt/local/lib/liblz4.1.9.4.dylib
       0x108a39000 -        0x108a40fff libb2.1.dylib (*) <0560357e-4a5d-39b9-87c2-456b31b8eb86> /opt/local/lib/libb2.1.dylib
       0x10aaa8000 -        0x10ab7bfff libxml2.2.dylib (*) <caad2c42-4285-3d76-9c80-4836f3980386> /opt/local/lib/libxml2.2.dylib
       0x10ae62000 -        0x10afe9fff libicui18n.74.2.dylib (*) <890d4e26-14f7-3bfd-8830-834afdbd8ecb> /opt/local/lib/libicui18n.74.2.dylib
       0x10b112000 -        0x10b23dfff libicuuc.74.2.dylib (*) <6740f319-9f3c-3f83-a141-ce33c79f55bc> /opt/local/lib/libicuuc.74.2.dylib
       0x10d023000 -        0x10ed82fff libicudata.74.2.dylib (*) <8ec9a33c-2c4d-334f-928a-96c8774f790f> /opt/local/lib/libicudata.74.2.dylib
       0x108551000 -        0x108554fff mmap.cpython-312-darwin.so (*) <25fdf2bc-9380-3cf9-be10-ebffe0b54331> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/mmap.cpython-312-darwin.so
       0x108a11000 -        0x108a2cfff libmagic.1.dylib (*) <9bad6e67-0756-314d-8818-39ff6ed3e940> /opt/local/lib/libmagic.1.dylib
       0x109294000 -        0x109327fff libzstd.1.5.6.dylib (*) <ee8538ea-ec8a-3f2c-91b4-24fcbc3977fb> /opt/local/lib/libzstd.1.5.6.dylib
       0x1089b2000 -        0x1089c3fff _ctypes.cpython-312-darwin.so (*) <72ea5b04-1a66-36b1-b4ac-4283c6452089> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_ctypes.cpython-312-darwin.so
       0x1089d5000 -        0x1089dcfff libffi.8.dylib (*) <36b79279-46ab-3794-849c-d0edb954693f> /opt/local/lib/libffi.8.dylib
       0x1090dc000 -        0x1091eafff unicodedata.cpython-312-darwin.so (*) <2232235d-fe4d-3191-911d-edb366323a52> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/unicodedata.cpython-312-darwin.so
       0x1083e7000 -        0x1083e7fff _opcode.cpython-312-darwin.so (*) <ed02cc38-5e87-3c55-8990-565394d96401> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_opcode.cpython-312-darwin.so
       0x108495000 -        0x108499fff binascii.cpython-312-darwin.so (*) <3683df63-35f0-302a-897c-46a2324f42a1> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/binascii.cpython-312-darwin.so
       0x1083d9000 -        0x1083dafff _queue.cpython-312-darwin.so (*) <f1812230-9049-3d6c-9fa7-40b932fe016c> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_queue.cpython-312-darwin.so
       0x1084ce000 -        0x1084d5fff array.cpython-312-darwin.so (*) <20fe8a33-6c42-31f0-ac9e-dc9cf1c38a6f> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/array.cpython-312-darwin.so
       0x108533000 -        0x108540fff _socket.cpython-312-darwin.so (*) <03a79cad-7e03-300a-947f-cd73794cc442> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_socket.cpython-312-darwin.so
       0x108771000 -        0x108785fff _pickle.cpython-312-darwin.so (*) <8950711a-abfd-314e-bb95-893e70c37a86> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_pickle.cpython-312-darwin.so
       0x1084c1000 -        0x1084c6fff _struct.cpython-312-darwin.so (*) <dcc07268-f23e-303d-bde8-656c11b8b0da> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_struct.cpython-312-darwin.so
       0x1084b5000 -        0x1084bbfff _blake2.cpython-312-darwin.so (*) <13b1960b-a18d-32b8-a569-80c66e484113> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_blake2.cpython-312-darwin.so
       0x1084a5000 -        0x1084abfff _hashlib.cpython-312-darwin.so (*) <91b8dfc2-7d96-3249-8382-d9187607778f> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_hashlib.cpython-312-darwin.so
       0x10951c000 -        0x109847fff libcrypto.3.dylib (*) <01f51df9-3af3-3f63-9649-951007d7ea42> /opt/local/libexec/*/libcrypto.3.dylib
       0x1083f1000 -        0x1083f5fff select.cpython-312-darwin.so (*) <7fb1bddf-f4d5-3fc8-aa2a-ca2f177f005e> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/select.cpython-312-darwin.so
       0x1083e0000 -        0x1083e2fff _posixsubprocess.cpython-312-darwin.so (*) <16853a68-4c10-3eb9-bc92-35c6efcd806d> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_posixsubprocess.cpython-312-darwin.so
       0x10837d000 -        0x10837ffff fcntl.cpython-312-darwin.so (*) <ead4b123-f57c-3da9-9b40-6443d857179e> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/fcntl.cpython-312-darwin.so
       0x108384000 -        0x108387fff _heapq.cpython-312-darwin.so (*) <9f734ca6-24b6-37b1-80fb-26af92f277c7> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_heapq.cpython-312-darwin.so
       0x1083ce000 -        0x1083d3fff _json.cpython-312-darwin.so (*) <e2b0b848-ba41-3755-a24f-e0c4a3e97f32> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_json.cpython-312-darwin.so
       0x108475000 -        0x108487fff _curses.cpython-312-darwin.so (*) <3fe1b8bd-79f7-319c-8c05-c765f5e1a43e> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_curses.cpython-312-darwin.so
       0x1084e4000 -        0x108520fff libncurses.6.dylib (*) <dd0cd6d7-f512-3e01-9ad8-c124d222eb6c> /opt/local/lib/libncurses.6.dylib
       0x10838c000 -        0x108395fff _sha2.cpython-312-darwin.so (*) <92862f99-320f-3057-bff8-cf400145bae7> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_sha2.cpython-312-darwin.so
       0x108333000 -        0x108334fff _random.cpython-312-darwin.so (*) <d12ae3ca-898f-3344-afad-0dfec196fbb7> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_random.cpython-312-darwin.so
       0x108359000 -        0x10835bfff _bisect.cpython-312-darwin.so (*) <ec5d5bd4-c66a-3143-af99-a1764cd86bc3> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_bisect.cpython-312-darwin.so
       0x1083bc000 -        0x1083c6fff math.cpython-312-darwin.so (*) <2a3a750b-f558-3866-a309-c1cb4b3fb32e> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/math.cpython-312-darwin.so
       0x10834e000 -        0x108353fff _lzma.cpython-312-darwin.so (*) <7cd66b36-32e5-3f8d-a1fc-9d01cc6b449b> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_lzma.cpython-312-darwin.so
       0x108441000 -        0x108464fff liblzma.5.dylib (*) <0ca15e4a-e6de-3527-a0fd-e58795a0050e> /opt/local/lib/liblzma.5.dylib
       0x108347000 -        0x108349fff _bz2.cpython-312-darwin.so (*) <759028c9-e080-3bda-a6cf-9fafcc010dac> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_bz2.cpython-312-darwin.so
       0x10839c000 -        0x1083affff libbz2.1.0.8.dylib (*) <ad83246e-d77e-3baa-b1de-f32a3dd2b451> /opt/local/lib/libbz2.1.0.8.dylib
       0x10833a000 -        0x108340fff zlib.cpython-312-darwin.so (*) <034f4532-cf9a-35c0-89df-a88990a26fac> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/zlib.cpython-312-darwin.so
       0x108361000 -        0x108374fff libz.1.3.1.dylib (*) <6547b625-57be-3910-b55e-b719eee2ccee> /opt/local/lib/libz.1.3.1.dylib
       0x108a8e000 -        0x108db9fff org.python.python (3.12.4, (c) 2001-2023 Python Software Foundation.) <249baf03-d9dc-30a2-8c2a-9641bbff2cda> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Python
       0x108301000 -        0x108318fff libintl.8.dylib (*) <8503304b-6a8a-352e-a6a2-ea1065f969ff> /opt/local/lib/libintl.8.dylib
       0x108559000 -        0x10865cfff libiconv.2.dylib (*) <b9d201c0-9016-3263-a13f-b53932a23e2c> /opt/local/lib/libiconv.2.dylib
       0x107ec6000 -        0x107ec7fff org.python.python (3.12.4) <e9d42889-4391-3528-8f89-020f12fa51c2> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
    0x7ff80ff4b000 -     0x7ff80ffdbb9f dyld (*) <baa6f02e-dff3-3562-8c99-ea2820c91ad9> /usr/lib/dyld
               0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=355.9M resident=0K(0%) swapped_out_or_unallocated=355.9M(100%)
Writable regions: Total=2.1G written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=2.1G(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        1 
MALLOC                             2.1G       47 
MALLOC guard page                   24K        6 
STACK GUARD                          4K        1 
Stack                             16.0M        1 
VM_ALLOCATE                       14.0M       15 
__DATA                            4937K      189 
__DATA_CONST                      7636K      151 
__DATA_DIRTY                       343K       58 
__LINKEDIT                       184.6M       52 
__OBJC_RO                         71.8M        1 
__OBJC_RW                         2200K        3 
__TEXT                           171.2M      206 
dyld private memory                260K        2 
mapped file                       40.6M        3 
shared memory                       28K        4 
===========                     =======  ======= 
TOTAL                              2.6G      740

Using a debug build (--with-pydebug --with-assertions --with-address-sanitizer --with-undefined-behavior-sanitizer) gives this additional output:

Debug memory block at address p=0x6080008747b0: API 'o'
    65 bytes originally requested
    The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
    The 8 pad bytes at tail=0x6080008747f1 are not all FORBIDDENBYTE (0xfd):
        at tail+0: 0x00 *** OUCH
        at tail+1: 0x00 *** OUCH
        at tail+2: 0x00 *** OUCH
        at tail+3: 0xfd
        at tail+4: 0xfd
        at tail+5: 0xfd
        at tail+6: 0xfd
        at tail+7: 0xfd
    Data at p: 00 00 00 00 00 00 00 00 ... 00 00 00 00 00 00 00 00

Enable tracemalloc to get the memory block allocation traceback

Fatal Python error: _PyMem_DebugRawFree: bad trailing pad byte
Python runtime state: initialized

Current thread 0x00007ff85381dfc0 (most recent call first):
  File "<shim>", line ??? in <interpreter trampoline>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 66 in describe_pyc
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 52 in compare_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 433 in _compare_using_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197 in compare_pair
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467 in _compare_using_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197 in compare_pair
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467 in _compare_using_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 69 in compare_root_paths
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 718 in run_diffoscope
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 767 in main
  File "/opt/local/bin/diffoscope", line 8 in <module>

The crash report when using the debug build:

Process:               Python [57513]
Path:                  /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
Identifier:            org.python.python
Version:               3.12.4 (3.12.4)
Code Type:             X86-64 (Native)
Parent Process:        zsh [826]
Responsible:           Terminal [593]
User ID:               501

Date/Time:             2024-06-28 12:49:57.8688 +1000
OS Version:            macOS 14.5 (23F79)
Report Version:        12
Bridge OS Version:     8.5 (21P5077)
Anonymous UUID:        1535D773-66C6-A888-8912-6C090207B7D2

Time Awake Since Boot: 210000 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000

Termination Reason:    Namespace SIGNAL, Code 6 Abort trap: 6
Terminating Process:   Python [57513]

ID   Vend/Dev
6318 73ff1002
Seconds Ago   ID   Type
   210000.0   6318 Attach

Application Specific Information:
abort() called

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib              0x7ff8102a414a __pthread_kill + 10
1   libsystem_pthread.dylib             0x7ff8102dcebd pthread_kill + 262
2   libsystem_c.dylib                   0x7ff810202a79 abort + 126
3   Python                                 0x10e1c731f fatal_error_exit + 31 (pylifecycle.c:2737)
4   Python                                 0x10e1c702c fatal_error + 876 (pylifecycle.c:2918)
5   Python                                 0x10e1c5f51 _Py_FatalErrorFunc + 129 (pylifecycle.c:2934)
6   Python                                 0x10da4bf57 _PyMem_DebugCheckAddress + 775 (obmalloc.c:2349)
7   Python                                 0x10da4ba71 _PyMem_DebugRawFree + 241 (obmalloc.c:2159)
8   Python                                 0x10da4ccf9 _PyMem_DebugFree + 41 (obmalloc.c:2296)
9   Python                                 0x10da4b1ea PyObject_Free + 570 (obmalloc.c:830)
10  Python                                 0x10daaa9c9 object_dealloc + 185 (typeobject.c:5516)
11  Python                                 0x10da43fe4 _Py_Dealloc + 420 (object.c:2625)
12  Python                                 0x10d8635c8 Py_DECREF + 440 (object.h:690)
13  Python                                 0x10d847f9c Py_XDECREF + 44 (object.h:798)
14  Python                                 0x10d85acd1 code_dealloc + 3409 (codeobject.c:1735)
15  Python                                 0x10da43fe4 _Py_Dealloc + 420 (object.c:2625)
16  Python                                 0x10da89bd8 Py_DECREF + 440 (object.h:690)
17  Python                                 0x10da88a9c Py_XDECREF + 44 (object.h:798)
18  Python                                 0x10da8a02d tupledealloc + 637 (tupleobject.c:206)
19  Python                                 0x10da43fe4 _Py_Dealloc + 420 (object.c:2625)
20  Python                                 0x10d8635c8 Py_DECREF + 440 (object.h:690)
21  Python                                 0x10d847f9c Py_XDECREF + 44 (object.h:798)
22  Python                                 0x10d85a4e4 code_dealloc + 1380 (codeobject.c:1725)
23  Python                                 0x10da43fe4 _Py_Dealloc + 420 (object.c:2625)
24  Python                                 0x10e0a1e28 Py_DECREF + 440 (object.h:690)
25  Python                                 0x10e0a1c5c Py_XDECREF + 44 (object.h:798)
26  Python                                 0x10e0a25d4 _PyFrame_ClearLocals + 852 (frame.c:125)
27  Python                                 0x10e0a2d2f _PyFrame_ClearExceptCode + 1503 (frame.c:150)
28  Python                                 0x10def0144 clear_gen_frame + 2260 (ceval.c:1565)
29  Python                                 0x10dedd44b _PyEvalFrameClearAndPop + 235 (ceval.c:1577)
30  Python                                 0x10de73cb2 _PyEval_EvalFrameDefault + 113346 (bytecodes.c:686)
31  Python                                 0x10d8e1051 _PyEval_EvalFrame + 353 (pycore_ceval.h:89)
32  Python                                 0x10d8dfde7 gen_send_ex2 + 3415 (genobject.c:230)
33  Python                                 0x10d8d6675 gen_iternext + 373 (genobject.c:605)
34  Python                                 0x10d93e15d list_extend + 2285 (listobject.c:944)
35  Python                                 0x10d93d85d _PyList_Extend + 29 (listobject.c:982)
36  Python                                 0x10d7bf145 PySequence_List + 101 (abstract.c:2121)
37  Python                                 0x10d7bf30a PySequence_Fast + 346 (abstract.c:2152)
38  Python                                 0x10dbb2093 PyUnicode_Join + 35 (unicodeobject.c:9547)
39  Python                                 0x10dc7fecd unicode_join + 29 (unicodeobject.c:11727)
40  Python                                 0x10dec8902 _PyEval_EvalFrameDefault + 460562 (bytecodes.c:3093)
41  Python                                 0x10de57ba1 _PyEval_EvalFrame + 353 (pycore_ceval.h:89)
42  Python                                 0x10de5778d _PyEval_Vector + 477 (ceval.c:1683)
43  Python                                 0x10d832b08 _PyFunction_Vectorcall + 744 (call.c:419)
44  Python                                 0x10d831dcd _PyVectorcall_Call + 669 (call.c:271)
45  Python                                 0x10d83239f _PyObject_Call + 495 (call.c:354)
46  Python                                 0x10d832532 PyObject_Call + 50 (call.c:379)
47  Python                                 0x10e523413 starmap_next + 675 (itertoolsmodule.c:1979)
48  Python                                 0x10de3b375 filter_next + 725 (bltinmodule.c:583)
49  Python                                 0x10d93e15d list_extend + 2285 (listobject.c:944)
50  Python                                 0x10dec8902 _PyEval_EvalFrameDefault + 460562 (bytecodes.c:3093)
51  Python                                 0x10de57ba1 _PyEval_EvalFrame + 353 (pycore_ceval.h:89)
52  Python                                 0x10de5778d _PyEval_Vector + 477 (ceval.c:1683)
53  Python                                 0x10d832b08 _PyFunction_Vectorcall + 744 (call.c:419)
54  Python                                 0x10d831dcd _PyVectorcall_Call + 669 (call.c:271)
55  Python                                 0x10d83239f _PyObject_Call + 495 (call.c:354)
56  Python                                 0x10d832532 PyObject_Call + 50 (call.c:379)
57  Python                                 0x10e523413 starmap_next + 675 (itertoolsmodule.c:1979)
58  Python                                 0x10de3b375 filter_next + 725 (bltinmodule.c:583)
59  Python                                 0x10d93e15d list_extend + 2285 (listobject.c:944)
60  Python                                 0x10d883ed4 method_vectorcall_O + 532 (descrobject.c:482)
61  Python                                 0x10d82f0a5 _PyObject_VectorcallTstate + 421 (pycore_call.h:92)
62  Python                                 0x10d83212a PyObject_Vectorcall + 58 (call.c:325)
63  Python                                 0x10debac07 _PyEval_EvalFrameDefault + 403991 (bytecodes.c:2714)
64  Python                                 0x10de57ba1 _PyEval_EvalFrame + 353 (pycore_ceval.h:89)
65  Python                                 0x10de5778d _PyEval_Vector + 477 (ceval.c:1683)
66  Python                                 0x10de5746f PyEval_EvalCode + 1375 (ceval.c:578)
67  Python                                 0x10e281b79 run_eval_code_obj + 681 (pythonrun.c:1722)
68  Python                                 0x10e27706a run_mod + 202 (pythonrun.c:1743)
69  Python                                 0x10e2739bf pyrun_file + 191 (pythonrun.c:1643)
70  Python                                 0x10e27140e _PyRun_SimpleFileObject + 1230 (pythonrun.c:433)
71  Python                                 0x10e270409 _PyRun_AnyFileObject + 185 (pythonrun.c:78)
72  Python                                 0x10e37ecb5 pymain_run_file_obj + 1173 (main.c:360)
73  Python                                 0x10e37d37e pymain_run_file + 670 (main.c:379)
74  Python                                 0x10e37aeff pymain_run_python + 5407 (main.c:629)
75  Python                                 0x10e379786 Py_RunMain + 278 (main.c:709)
76  Python                                 0x10e37b7f5 pymain_main + 757 (main.c:739)
77  Python                                 0x10e37ba8e Py_BytesMain + 494 (main.c:763)
78  Python                                 0x10a6aef72 0x10a6ad000 + 8050
79  dyld                                0x7ff80ff51366 start + 1942

Thread 0 crashed with X86 Thread State (64-bit):
  rax: 0x0000000000000000  rbx: 0x0000000000000006  rcx: 0x00007ff7b581fa18  rdx: 0x0000000000000000
  rdi: 0x0000000000000103  rsi: 0x0000000000000006  rbp: 0x00007ff7b581fa40  rsp: 0x00007ff7b581fa18
   r8: 0x00001ffef6b03f48   r9: 0x0000000000000000  r10: 0x0000000000000000  r11: 0x0000000000000246
  r12: 0x0000000000000103  r13: 0x0000000000000000  r14: 0x00007ff85381dfc0  r15: 0x0000000000000016
  rip: 0x00007ff8102a414a  rfl: 0x0000000000000246  cr2: 0x0000000000000000

Logical CPU:     0
Error Code:      0x02000148 
Trap Number:     133

Binary Images:
       0x1115ef000 -        0x11163efff _elementtree.cpython-312d-darwin.so (*) <87abd5ba-ce9c-39a6-af56-ed4cc6cb290e> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_elementtree.cpython-312d-darwin.so
       0x111785000 -        0x1117dbfff _datetime.cpython-312d-darwin.so (*) <2fe1439c-492a-3eaf-9f51-9b8f90add038> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_datetime.cpython-312d-darwin.so
       0x10d243000 -        0x10d264fff pyexpat.cpython-312d-darwin.so (*) <ae8ef261-4c9d-37fc-8cba-5926a9051ac3> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/pyexpat.cpython-312d-darwin.so
       0x10d10e000 -        0x10d129fff libexpat.1.9.2.dylib (*) <f4891366-5776-3908-8ebb-4725998434c3> /opt/local/lib/libexpat.1.9.2.dylib
       0x10a7a7000 -        0x10a7a8fff _uuid.cpython-312d-darwin.so (*) <86bf351c-d8be-34f2-88ed-167579126151> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_uuid.cpython-312d-darwin.so
       0x10d151000 -        0x10d1dcfff libarchive.13.dylib (*) <88517cd2-ef65-3864-8983-930d441f34b0> /opt/local/lib/libarchive.13.dylib
       0x10cd24000 -        0x10cd3ffff liblzo2.2.dylib (*) <305437be-f124-365c-9d7c-e9d236bd10a6> /opt/local/lib/liblzo2.2.dylib
       0x10ce2a000 -        0x10ce49fff liblz4.1.9.4.dylib (*) <28e8eabe-f9ef-3635-9796-8551550c48f9> /opt/local/lib/liblz4.1.9.4.dylib
       0x10c7e0000 -        0x10c7e7fff libb2.1.dylib (*) <0560357e-4a5d-39b9-87c2-456b31b8eb86> /opt/local/lib/libb2.1.dylib
       0x111679000 -        0x11174cfff libxml2.2.dylib (*) <caad2c42-4285-3d76-9c80-4836f3980386> /opt/local/lib/libxml2.2.dylib
       0x111a33000 -        0x111bbafff libicui18n.74.2.dylib (*) <890d4e26-14f7-3bfd-8830-834afdbd8ecb> /opt/local/lib/libicui18n.74.2.dylib
       0x111ce3000 -        0x111e0efff libicuuc.74.2.dylib (*) <6740f319-9f3c-3f83-a141-ce33c79f55bc> /opt/local/lib/libicuuc.74.2.dylib
       0x113bf4000 -        0x115953fff libicudata.74.2.dylib (*) <8ec9a33c-2c4d-334f-928a-96c8774f790f> /opt/local/lib/libicudata.74.2.dylib
       0x10cd01000 -        0x10cd14fff mmap.cpython-312d-darwin.so (*) <a48ee8e9-d1ec-32e9-9db5-b1733a004e36> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/mmap.cpython-312d-darwin.so
       0x10cb4c000 -        0x10cb67fff libmagic.1.dylib (*) <9bad6e67-0756-314d-8818-39ff6ed3e940> /opt/local/lib/libmagic.1.dylib
       0x10cd82000 -        0x10ce15fff libzstd.1.5.6.dylib (*) <ee8538ea-ec8a-3f2c-91b4-24fcbc3977fb> /opt/local/lib/libzstd.1.5.6.dylib
       0x10cfd4000 -        0x10d052fff _ctypes.cpython-312d-darwin.so (*) <5d336ea2-2f33-3329-9476-eed227e9d809> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_ctypes.cpython-312d-darwin.so
       0x10c738000 -        0x10c73ffff libffi.8.dylib (*) <36b79279-46ab-3794-849c-d0edb954693f> /opt/local/lib/libffi.8.dylib
       0x10ce59000 -        0x10cfbafff unicodedata.cpython-312d-darwin.so (*) <fbb8d225-6e9f-3eec-a179-3842b76db1c3> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/unicodedata.cpython-312d-darwin.so
       0x10a7a0000 -        0x10a7a2fff _opcode.cpython-312d-darwin.so (*) <a2f26edf-0278-3d4c-8b57-411449f5a17b> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_opcode.cpython-312d-darwin.so
       0x10c715000 -        0x10c728fff binascii.cpython-312d-darwin.so (*) <719d872d-b3dc-34e1-ac7d-f02e2ed989fc> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/binascii.cpython-312d-darwin.so
       0x10a9bc000 -        0x10a9c4fff _queue.cpython-312d-darwin.so (*) <d55f3e31-de43-3428-be53-94c2c2e27274> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_queue.cpython-312d-darwin.so
       0x10cb00000 -        0x10cb2bfff array.cpython-312d-darwin.so (*) <dc06a53c-79c0-34c3-8367-9e0fa0081397> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/array.cpython-312d-darwin.so
       0x10cc6e000 -        0x10ccabfff _socket.cpython-312d-darwin.so (*) <891cd0f6-b18a-3704-a1bf-0701e891c331> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_socket.cpython-312d-darwin.so
       0x10cbb7000 -        0x10cc2efff _pickle.cpython-312d-darwin.so (*) <94714b05-2e0d-3891-8e6c-ddd6ab5b65f4> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_pickle.cpython-312d-darwin.so
       0x10afbd000 -        0x10afddfff _struct.cpython-312d-darwin.so (*) <19c9421a-6863-35ba-a0ad-5a3727516ff5> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_struct.cpython-312d-darwin.so
       0x10c758000 -        0x10c786fff _blake2.cpython-312d-darwin.so (*) <52b31d9e-add3-35bf-ae60-b0c97596d5af> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_blake2.cpython-312d-darwin.so
       0x10aff7000 -        0x10b013fff _hashlib.cpython-312d-darwin.so (*) <22b4a6c9-25bf-359f-b427-e0d0f6c64775> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_hashlib.cpython-312d-darwin.so
       0x1100ed000 -        0x110418fff libcrypto.3.dylib (*) <01f51df9-3af3-3f63-9649-951007d7ea42> /opt/local/libexec/*/libcrypto.3.dylib
       0x10aaab000 -        0x10aac1fff select.cpython-312d-darwin.so (*) <dbae4518-a6f9-3f84-bb67-6b7061e8546f> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/select.cpython-312d-darwin.so
       0x10a996000 -        0x10a99efff _posixsubprocess.cpython-312d-darwin.so (*) <e9c67967-2a5b-3df5-821a-d7b27fcb2181> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_posixsubprocess.cpython-312d-darwin.so
       0x10a7fd000 -        0x10a803fff fcntl.cpython-312d-darwin.so (*) <a99a672c-41a4-3609-baca-50d939c44454> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/fcntl.cpython-312d-darwin.so
       0x10a85d000 -        0x10a865fff _heapq.cpython-312d-darwin.so (*) <6e35a5b4-3741-3518-8312-2e96e18778f9> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_heapq.cpython-312d-darwin.so
       0x10a966000 -        0x10a982fff _json.cpython-312d-darwin.so (*) <b909179f-c49c-3652-986a-102875b402cf> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_json.cpython-312d-darwin.so
       0x10a9e1000 -        0x10aa20fff _curses.cpython-312d-darwin.so (*) <12f0eefc-92a5-3342-8bd9-98db89abe04b> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_curses.cpython-312d-darwin.so
       0x10aa5c000 -        0x10aa98fff libncurses.6.dylib (*) <dd0cd6d7-f512-3e01-9ad8-c124d222eb6c> /opt/local/lib/libncurses.6.dylib
       0x10ae71000 -        0x10af65fff _sha2.cpython-312d-darwin.so (*) <05dfe7f4-95c2-3f8d-9cd2-67e72fc707cd> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_sha2.cpython-312d-darwin.so
       0x10a84c000 -        0x10a853fff _random.cpython-312d-darwin.so (*) <48a1c81f-5c08-3369-97a2-1f719b9dbdee> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_random.cpython-312d-darwin.so
       0x10a7ee000 -        0x10a7f5fff _bisect.cpython-312d-darwin.so (*) <9af63420-3feb-31ad-9338-436166beee99> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_bisect.cpython-312d-darwin.so
       0x10a924000 -        0x10a94bfff math.cpython-312d-darwin.so (*) <dd7df02d-f682-3937-bbbf-b9482d434028> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/math.cpython-312d-darwin.so
       0x10a87d000 -        0x10a897fff _lzma.cpython-312d-darwin.so (*) <85e29cab-5be5-3674-b123-d7dc70a23ca0> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_lzma.cpython-312d-darwin.so
       0x10a8ae000 -        0x10a8d1fff liblzma.5.dylib (*) <0ca15e4a-e6de-3527-a0fd-e58795a0050e> /opt/local/lib/liblzma.5.dylib
       0x10a80d000 -        0x10a81dfff _bz2.cpython-312d-darwin.so (*) <5521d60e-ead5-384b-93db-0216cba0b0f1> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/_bz2.cpython-312d-darwin.so
       0x10a82c000 -        0x10a83ffff libbz2.1.0.8.dylib (*) <ad83246e-d77e-3baa-b1de-f32a3dd2b451> /opt/local/lib/libbz2.1.0.8.dylib
       0x10a7ac000 -        0x10a7d2fff zlib.cpython-312d-darwin.so (*) <62b47845-1e2f-358d-acd3-a7c73f17a493> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/lib-dynload/zlib.cpython-312d-darwin.so
       0x10a784000 -        0x10a797fff libz.1.3.1.dylib (*) <6547b625-57be-3910-b55e-b719eee2ccee> /opt/local/lib/libz.1.3.1.dylib
       0x10d296000 -        0x10e95dfff org.python.python (3.12.4, (c) 2001-2023 Python Software Foundation.) <cc749d46-4066-3e71-be89-8dc679dd76c6> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Python
       0x10a6d7000 -        0x10a6eefff libintl.8.dylib (*) <8503304b-6a8a-352e-a6a2-ea1065f969ff> /opt/local/lib/libintl.8.dylib
       0x10b03c000 -        0x10b16afff libclang_rt.asan_osx_dynamic.dylib (*) <b6625699-be3f-3ec9-a61f-e185dfe07d8c> /Library/Developer/CommandLineTools/usr/lib/clang/15.0.0/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
       0x10ad59000 -        0x10ae5cfff libiconv.2.dylib (*) <b9d201c0-9016-3263-a13f-b53932a23e2c> /opt/local/lib/libiconv.2.dylib
       0x10a6ad000 -        0x10a6aefff org.python.python (3.12.4) <8edc560b-69b2-328a-9833-fade1a70eed1> /opt/local/Library/Frameworks/Python.framework/Versions/3.12/Resources/Python.app/Contents/MacOS/Python
    0x7ff81029c000 -     0x7ff8102d6ff7 libsystem_kernel.dylib (*) <13597eeb-2ea0-3d48-8be1-dfc4872a784b> /usr/lib/system/libsystem_kernel.dylib
    0x7ff8102d7000 -     0x7ff8102e2ff7 libsystem_pthread.dylib (*) <25b2f3eb-07b4-3cb9-9457-2c26210c27c8> /usr/lib/system/libsystem_pthread.dylib
    0x7ff810183000 -     0x7ff81020aff7 libsystem_c.dylib (*) <93fb3816-608b-33af-83d2-209b4bd673ad> /usr/lib/system/libsystem_c.dylib
    0x7ff80ff4b000 -     0x7ff80ffdbb9f dyld (*) <baa6f02e-dff3-3562-8c99-ea2820c91ad9> /usr/lib/dyld
               0x0 - 0xffffffffffffffff ??? (*) <00000000-0000-0000-0000-000000000000> ???

External Modification Summary:
  Calls made by other processes targeting this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by this process:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0
  Calls made by all processes on this machine:
    task_for_pid: 0
    thread_create: 0
    thread_set_state: 0

VM Region Summary:
ReadOnly portion of Libraries: Total=387.3M resident=0K(0%) swapped_out_or_unallocated=387.3M(100%)
Writable regions: Total=14.0T written=0K(0%) resident=0K(0%) swapped_out=0K(0%) unallocated=14.0T(100%)

                                VIRTUAL   REGION 
REGION TYPE                        SIZE    COUNT (non-coalesced) 
===========                     =======  ======= 
Kernel Alloc Once                    8K        1 
MALLOC                             132K        6 
MALLOC guard page                   24K        6 
STACK GUARD                          4K        1 
Sanitizer                          6.0T     4663 
Sanitizer (reserved)              14.0T      128         reserved VM address space (unallocated)
Stack                             64.0M        1 
__DATA                            23.5M      191 
__DATA_CONST                      7772K      152 
__DATA_DIRTY                       343K       58 
__LINKEDIT                       190.9M       53 
__OBJC_RO                         71.8M        1 
__OBJC_RW                         2200K        3 
__TEXT                           196.4M      207 
dyld private memory                260K        2 
mapped file                       40.6M        4 
shared memory                       28K        4 
===========                     =======  ======= 
TOTAL                             20.0T     5481 
TOTAL, minus reserved VM space     6.0T     5481

The two .tbz2 files are attached (in a zip file to appease github.) Testcase.zip

CPython versions tested on:

3.12

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

Python 3.12.4 (main, Jun 8 2024, 03:35:50) [Clang 15.0.0 (clang-1500.3.9.4)]

jmroot commented 2 months ago

Output with PYTHONTRACEMALLOC=32 set (this takes quite a while to run):

Debug memory block at address p=0x6080002ef8b0: API 'o'
    65 bytes originally requested
    The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
    The 8 pad bytes at tail=0x6080002ef8f1 are not all FORBIDDENBYTE (0xfd):
        at tail+0: 0x00 *** OUCH
        at tail+1: 0x00 *** OUCH
        at tail+2: 0x00 *** OUCH
        at tail+3: 0xfd
        at tail+4: 0xfd
        at tail+5: 0xfd
        at tail+6: 0xfd
        at tail+7: 0xfd
    Data at p: 00 00 00 00 00 00 00 00 ... 00 00 00 00 00 00 00 00

Memory block allocated at (most recent call first):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 100
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 109
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 89
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 66
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 52
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 433
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 69
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 718
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 767
  File "/opt/local/bin/diffoscope", line 8

Fatal Python error: _PyMem_DebugRawFree: bad trailing pad byte
Python runtime state: initialized

Current thread 0x00007ff85381dfc0 (most recent call first):
  File "<shim>", line ??? in <interpreter trampoline>
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 66 in describe_pyc
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 52 in compare_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 433 in _compare_using_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197 in compare_pair
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467 in _compare_using_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197 in compare_pair
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467 in _compare_using_details
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 69 in compare_root_paths
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 718 in run_diffoscope
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 767 in main
  File "/opt/local/bin/diffoscope", line 8 in <module>
vstinner commented 3 weeks ago

It looks like a buffer overflow in a C extension used by diffoscope, rather than a bug in Python. I suggest opening a bug report in diffoscope instead.

jmroot commented 3 weeks ago

Thanks for taking a look. I had considered that possibility, but so far I've had no luck catching the bad write using ASAN or libgmalloc. I think the only non-stdlib extensions in use should be libarchive via ctypes. I will report to diffoscope as well and see if they can offer any clues.

emanuelb commented 3 weeks ago

I will report to diffoscope as well and see if they can offer any clues.

I opened issue upstream for diffoscope at : https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/388

The segfault is reproducible on Linux as well when running the diffoscope docker container, there are multiple .pyc files in /opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/tortoisehg/hgqt/__pycache__ directory that cause segfault when compared with diffoscope.

For example the files:

Thus its not related to archive operations as it can be reproduced when comparing .pyc files from the archives directly.

obfusk commented 3 weeks ago

I'm guessing this has the same cause as #118990 (https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/371).

jmroot commented 3 weeks ago

Thanks, I guess this is a duplicate one way or another then.

jmroot commented 3 weeks ago

I've now read through the diffoscope issues, and a significant part of the problem was that diffoscope was running marshal.load() on untrusted pyc files, and dis.disassemble() on code that could be from a different python version. A change to stop doing that has been committed.

However it still seems pretty bad that passing invalid data to these functions can cause out-of-bounds memory writes, particularly when it's not malicious but just compiled with a (slightly) older python version.

obfusk commented 3 weeks ago

Yeah, as I said in https://github.com/python/cpython/issues/118990#issuecomment-2156275445:

According to https://github.com/python/cpython/issues/113626:

The format of the code objects changed with every Python version, and this is not reflected in marshal format version. Loading marshal data created in different Python version has undefined behavior if the data contains a code object.

Which answers the question of whether this is supported (though I'm still unsure whether it should crash like this, though I supposed "undefined behavior" could include crashing).

vstinner commented 1 week ago

IMO diffoscope should have its own code to decode PYC files, rather than using Python marshal module. We only support loading PYC files of the same Python version.

vstinner commented 1 week ago

I suggest to close the issue since IMO it's a bug in a 3rd party project, not in Python.

jmroot commented 1 week ago

I suggest to close the issue since IMO it's a bug in a 3rd party project, not in Python.

I don't think anyone disagrees that diffoscope was misusing these functions, and it has already stopped doing so. But I also don't think it's unreasonable to ask for better behaviour than silent memory corruption when someone does try to load an unsupported PYC.

vstinner commented 1 week ago

But I also don't think it's unreasonable to ask for better behaviour than silent memory corruption when someone does try to load an unsupported PYC.

I would help to have reproducer which doesn't use diffoscope.

obfusk commented 1 week ago

I would help to have reproducer which doesn't use diffoscope.

https://github.com/python/cpython/issues/118990 has one that should work for this issue too (if you replace the embedded base64-encoded PYC with the one from this issue or just load it from a file instead).

vstinner commented 1 week ago

https://github.com/python/cpython/issues/118990 has one that should work for this issue too (if you replace the embedded base64-encoded PYC with the one from this issue or just load it from a file instead).

Please explain how to get the PYC file of this issue.

The reproducer of the issue gh-118990 fails on Python 3.14 on dis.disassemble() with:

Traceback (most recent call last):
  File "/home/vstinner/python/main/x.py", line 413, in <module>
    len(list(parse_pyc(io.BytesIO(decoded_string))))
        ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/x.py", line 362, in parse_pyc
    yield from show_code(code)
  File "/home/vstinner/python/main/x.py", line 375, in show_code
    dis.disassemble(code, file=s)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/dis.py", line 808, in disassemble
    _disassemble_bytes(_get_code_array(co, adaptive), lasti, linestarts,
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       exception_entries=exception_entries, co_positions=co.co_positions(),
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       original_code=co.co_code, arg_resolver=arg_resolver, formatter=formatter)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/dis.py", line 887, in _disassemble_bytes
    print_instructions(instrs, exception_entries, formatter, lasti=lasti)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/dis.py", line 891, in print_instructions
    for instr in instrs:
                 ^^^^^^
  File "/home/vstinner/python/main/Lib/dis.py", line 764, in _get_instructions_bytes
    argval, argrepr = arg_resolver.get_argval_argrepr(op, arg, offset)
                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/dis.py", line 601, in get_argval_argrepr
    val1, argrepr1 = _get_name_info(arg1, self.varname_from_oparg)
                     ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/vstinner/python/main/Lib/dis.py", line 699, in _get_name_info
    argval = get_name(name_index, **extrainfo)
IndexError: tuple index out of range
obfusk commented 1 week ago

Please explain how to get the PYC file of this issue.

From the test case .zip posted in the very first post of this issue?

vstinner commented 1 week ago

From the test case .zip posted in the very first post of this issue?

Testcase.zip contains 2 archives, which one should be used? tortoisehg-6.6.3_0.darwin_23.noarch.tbz2 contains 272 PYC files, which one should be used?

obfusk commented 1 week ago

Testcase.zip contains 2 archives, which one should be used? tortoisehg-6.6.3_0.darwin_23.noarch.tbz2 contains 272 PYC files, which one should be used?

Good question. Helpfully, we have this list of files for which there is a confirmed segfault on Linux as well: https://github.com/python/cpython/issues/121112#issuecomment-2279625164

AFAICT the actual differences -- at least for e.g. guess.cpython-311.pyc -- seem to be embedded timestamps, not bytecode, so I suspect either archive should work to reproduce.

But perhaps @jmroot can confirm a specific file to try?

jmroot commented 1 week ago

But perhaps @jmroot can confirm a specific file to try?

I'm afraid I don't have a reduced test case prepared that doesn't use diffoscope and only uses a single pyc file. I'm not sure when I will have time to prepare one.

This is the commit removing the code that was triggering the issue in diffoscope, if that helps: https://salsa.debian.org/reproducible-builds/diffoscope/-/commit/e75871b07e09cfd778181d905f540a15bd71e63a