Open jmroot opened 2 months ago
Output with PYTHONTRACEMALLOC=32
set (this takes quite a while to run):
Debug memory block at address p=0x6080002ef8b0: API 'o'
65 bytes originally requested
The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
The 8 pad bytes at tail=0x6080002ef8f1 are not all FORBIDDENBYTE (0xfd):
at tail+0: 0x00 *** OUCH
at tail+1: 0x00 *** OUCH
at tail+2: 0x00 *** OUCH
at tail+3: 0xfd
at tail+4: 0xfd
at tail+5: 0xfd
at tail+6: 0xfd
at tail+7: 0xfd
Data at p: 00 00 00 00 00 00 00 00 ... 00 00 00 00 00 00 00 00
Memory block allocated at (most recent call first):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 100
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 109
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 89
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 66
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 52
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 433
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 69
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 718
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 767
File "/opt/local/bin/diffoscope", line 8
Fatal Python error: _PyMem_DebugRawFree: bad trailing pad byte
Python runtime state: initialized
Current thread 0x00007ff85381dfc0 (most recent call first):
File "<shim>", line ??? in <interpreter trampoline>
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 66 in describe_pyc
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/python.py", line 52 in compare_details
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 433 in _compare_using_details
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197 in compare_pair
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467 in _compare_using_details
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/container.py", line 197 in compare_pair
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 467 in _compare_using_details
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/file.py", line 532 in compare
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 149 in compare_files
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/comparators/utils/compare.py", line 69 in compare_root_paths
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 718 in run_diffoscope
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/diffoscope/main.py", line 767 in main
File "/opt/local/bin/diffoscope", line 8 in <module>
It looks like a buffer overflow in a C extension used by diffoscope, rather than a bug in Python. I suggest opening a bug report in diffoscope instead.
Thanks for taking a look. I had considered that possibility, but so far I've had no luck catching the bad write using ASAN or libgmalloc. I think the only non-stdlib extensions in use should be libarchive via ctypes. I will report to diffoscope as well and see if they can offer any clues.
I will report to diffoscope as well and see if they can offer any clues.
I opened issue upstream for diffoscope at : https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/388
The segfault is reproducible on Linux as well when running the diffoscope docker container, there are multiple .pyc files in /opt/local/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/tortoisehg/hgqt/__pycache__
directory that cause segfault when compared with diffoscope.
For example the files:
Thus its not related to archive operations as it can be reproduced when comparing .pyc files from the archives directly.
I'm guessing this has the same cause as #118990 (https://salsa.debian.org/reproducible-builds/diffoscope/-/issues/371).
Thanks, I guess this is a duplicate one way or another then.
I've now read through the diffoscope issues, and a significant part of the problem was that diffoscope was running marshal.load()
on untrusted pyc files, and dis.disassemble()
on code that could be from a different python version. A change to stop doing that has been committed.
However it still seems pretty bad that passing invalid data to these functions can cause out-of-bounds memory writes, particularly when it's not malicious but just compiled with a (slightly) older python version.
Yeah, as I said in https://github.com/python/cpython/issues/118990#issuecomment-2156275445:
According to https://github.com/python/cpython/issues/113626:
The format of the code objects changed with every Python version, and this is not reflected in marshal format version. Loading marshal data created in different Python version has undefined behavior if the data contains a code object.
Which answers the question of whether this is supported (though I'm still unsure whether it should crash like this, though I supposed "undefined behavior" could include crashing).
IMO diffoscope should have its own code to decode PYC files, rather than using Python marshal module. We only support loading PYC files of the same Python version.
I suggest to close the issue since IMO it's a bug in a 3rd party project, not in Python.
I suggest to close the issue since IMO it's a bug in a 3rd party project, not in Python.
I don't think anyone disagrees that diffoscope was misusing these functions, and it has already stopped doing so. But I also don't think it's unreasonable to ask for better behaviour than silent memory corruption when someone does try to load an unsupported PYC.
But I also don't think it's unreasonable to ask for better behaviour than silent memory corruption when someone does try to load an unsupported PYC.
I would help to have reproducer which doesn't use diffoscope.
I would help to have reproducer which doesn't use diffoscope.
https://github.com/python/cpython/issues/118990 has one that should work for this issue too (if you replace the embedded base64-encoded PYC with the one from this issue or just load it from a file instead).
https://github.com/python/cpython/issues/118990 has one that should work for this issue too (if you replace the embedded base64-encoded PYC with the one from this issue or just load it from a file instead).
Please explain how to get the PYC file of this issue.
The reproducer of the issue gh-118990 fails on Python 3.14 on dis.disassemble()
with:
Traceback (most recent call last):
File "/home/vstinner/python/main/x.py", line 413, in <module>
len(list(parse_pyc(io.BytesIO(decoded_string))))
~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vstinner/python/main/x.py", line 362, in parse_pyc
yield from show_code(code)
File "/home/vstinner/python/main/x.py", line 375, in show_code
dis.disassemble(code, file=s)
~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
File "/home/vstinner/python/main/Lib/dis.py", line 808, in disassemble
_disassemble_bytes(_get_code_array(co, adaptive), lasti, linestarts,
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
exception_entries=exception_entries, co_positions=co.co_positions(),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
original_code=co.co_code, arg_resolver=arg_resolver, formatter=formatter)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vstinner/python/main/Lib/dis.py", line 887, in _disassemble_bytes
print_instructions(instrs, exception_entries, formatter, lasti=lasti)
~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vstinner/python/main/Lib/dis.py", line 891, in print_instructions
for instr in instrs:
^^^^^^
File "/home/vstinner/python/main/Lib/dis.py", line 764, in _get_instructions_bytes
argval, argrepr = arg_resolver.get_argval_argrepr(op, arg, offset)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
File "/home/vstinner/python/main/Lib/dis.py", line 601, in get_argval_argrepr
val1, argrepr1 = _get_name_info(arg1, self.varname_from_oparg)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/vstinner/python/main/Lib/dis.py", line 699, in _get_name_info
argval = get_name(name_index, **extrainfo)
IndexError: tuple index out of range
Please explain how to get the PYC file of this issue.
From the test case .zip posted in the very first post of this issue?
From the test case .zip posted in the very first post of this issue?
Testcase.zip contains 2 archives, which one should be used? tortoisehg-6.6.3_0.darwin_23.noarch.tbz2
contains 272 PYC files, which one should be used?
Testcase.zip contains 2 archives, which one should be used?
tortoisehg-6.6.3_0.darwin_23.noarch.tbz2
contains 272 PYC files, which one should be used?
Good question. Helpfully, we have this list of files for which there is a confirmed segfault on Linux as well: https://github.com/python/cpython/issues/121112#issuecomment-2279625164
AFAICT the actual differences -- at least for e.g. guess.cpython-311.pyc
-- seem to be embedded timestamps, not bytecode, so I suspect either archive should work to reproduce.
But perhaps @jmroot can confirm a specific file to try?
But perhaps @jmroot can confirm a specific file to try?
I'm afraid I don't have a reduced test case prepared that doesn't use diffoscope and only uses a single pyc file. I'm not sure when I will have time to prepare one.
This is the commit removing the code that was triggering the issue in diffoscope, if that helps: https://salsa.debian.org/reproducible-builds/diffoscope/-/commit/e75871b07e09cfd778181d905f540a15bd71e63a
Crash report
What happened?
Running diffoscope version 267 under Python 3.12.4, both installed via MacPorts on Intel macOS 14.5, segfaults when given two specific files as input.
The macOS crash report from the optimized build:
Using a debug build (
--with-pydebug --with-assertions --with-address-sanitizer --with-undefined-behavior-sanitizer
) gives this additional output:The crash report when using the debug build:
The two .tbz2 files are attached (in a zip file to appease github.) Testcase.zip
CPython versions tested on:
3.12
Operating systems tested on:
macOS
Output from running 'python -VV' on the command line:
Python 3.12.4 (main, Jun 8 2024, 03:35:50) [Clang 15.0.0 (clang-1500.3.9.4)]