libgit2 / pygit2

Python bindings for libgit2
https://www.pygit2.org/
Other
1.58k stars 382 forks source link

Work around size mismatch between enum and Py_ssize_t #1130

Closed roehling closed 2 years ago

roehling commented 2 years ago

The ODB backend class uses PyArgs_ParseTuple with format identifier "n" to parse git_object_t type IDs. However, enums are not necessarily of type Py_ssize_t; in fact, the C standard does not constrain the underlying integer type. On big endian architectures such as s390x, this leads to unwanted cutoffs of the lower bits, yielding 0 instead of 3 as object type in tests/test_odb_backend.py.

This PR works around the issue by parsing the type into a well-known integer type first, and then casting to the git_object_t enum type, avoiding the undefined behavior.