Closed mgorny closed 4 years ago
Thank you for the bug report. I'll try to have a look this evening.
Could you please add to the description:
$ python3.6 --version
Python 3.6.10
$ python3.7 --version
Python 3.7.6
$ python3.8 --version
Python 3.8.2
Gentoo Linux/amd64.
I could not reproduce the crash locally (Ubuntu 18.04 x64) against any of 3.6.9, 3.7.6 or 3.8.2.
However, the fact it fails on the line which attempts to PyBytes_GET_SIZE
- I think the length retrieval should be against raw
and not str
- that looks like it might be the bug. I'll try that tomorrow.
In the mean time - @mgorny might you be able to paste in the compiler arguments which were used to build the extension your end, one for any of the objects and final linking, e.g.:
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/vt/repos/iotic/py-ubjson/py3.8.2/include -I/usr/include/python3.8 -c src/python_funcs.c -o build/temp.linux-x86_64-3.8/src/python_funcs.o -std=c99
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/src/_ubjson.o build/temp.linux-x86_64-3.8/src/decoder.o build/temp.linux-x86_64-3.8/src/encoder.o build/temp.linux-x86_64-3.8/src/python_funcs.o -o /home/vt/repos/iotic/py-ubjson/_ubjson.cpython-38-x86_64-linux-gnu.so
Sure:
x86_64-pc-linux-gnu-gcc-9.2.0 -march=x86-64 -mtune=k8 -mcx16 -msahf -msse3 --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -O2 -pipe -frecord-gcc-switches -fPIC -I/usr/include/python3.6m -c src/_ubjson.c -o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/_ubjson.o -std=c99
x86_64-pc-linux-gnu-gcc-9.2.0 -march=x86-64 -mtune=k8 -mcx16 -msahf -msse3 --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -O2 -pipe -frecord-gcc-switches -fPIC -I/usr/include/python3.6m -c src/decoder.c -o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/decoder.o -std=c99
x86_64-pc-linux-gnu-gcc-9.2.0 -march=x86-64 -mtune=k8 -mcx16 -msahf -msse3 --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -O2 -pipe -frecord-gcc-switches -fPIC -I/usr/include/python3.6m -c src/encoder.c -o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/encoder.o -std=c99
x86_64-pc-linux-gnu-gcc-9.2.0 -march=x86-64 -mtune=k8 -mcx16 -msahf -msse3 --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -O2 -pipe -frecord-gcc-switches -fPIC -I/usr/include/python3.6m -c src/python_funcs.c -o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/python_funcs.o -std=c99
x86_64-pc-linux-gnu-gcc-9.2.0 -shared -Wl,-O1 -Wl,--as-needed -Wl,--defsym=__gentoo_check_ldflags__=0 -march=x86-64 -mtune=k8 -mcx16 -msahf -msse3 --param l1-cache-size=64 --param l1-cache-line-size=64 --param l2-cache-size=512 -O2 -pipe -frecord-gcc-switches /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/_ubjson.o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/decoder.o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/encoder.o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/temp.linux-x86_64-3.6/src/python_funcs.o -L/usr/lib64 -lpython3.6m -o /tmp/portage/dev-python/py-ubjson-0.14.0/work/py-ubjson-0.14.0-python3_6/lib/_ubjson.cpython-36m-x86_64-linux-gnu.so
If I were to guess, -DNDEBUG
is stripping away runtime checks (on your system).
Reproduced by adding undef_macros=['NDEBUG']
to extension definition and then using python setup.py build_ext -ig
@mgorny , might you be able to test the proposed fix in #13 ? (It fixes it for me with asserts enabled and from code inspection I think this is the only place where PyBytes_GET_SIZE
was mistakenly called against the wrong object.)
I can confirm that this patch fixes the crash for me, and all tests pass after applying it.
Fixed released in v0.15.0: https://pypi.org/project/py-ubjson/0.15.0/
The extension crashes while running the tests:
To reproduce: