search

doloopwhile / pyjq

A Python binding for ./jq
MIT License
196 stars 30 forks source link

Memory corruption when passing args hash to compile() #51

Closed mfeit-internet2 closed 3 years ago

mfeit-internet2 commented 4 years ago

In perfsonar/pscheduler#1059, it was discovered that this script...

import pyjq
f = pyjq.compile("$value", {"value": 123})

...causes Python 2 and 3 to crash with a memory corruption error:

$ ~/tmp/pyt
*** Error in `python3': malloc(): memory corruption (fast): 0x0000000001c5b38f ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7f3e4)[0x7fcdf600c3e4]
/lib64/libc.so.6(+0x82b20)[0x7fcdf600fb20]
/lib64/libc.so.6(__libc_calloc+0xb4)[0x7fcdf6013184]
/lib64/libjq.so.1(jv_mem_calloc+0x9)[0x7fcdeef32c19]
/lib64/libjq.so.1(+0x157ae)[0x7fcdeef287ae]
/lib64/libjq.so.1(+0x1607b)[0x7fcdeef2907b]
/lib64/libjq.so.1(jq_compile_args+0x30d)[0x7fcdeef2c12d]
/usr/lib64/python3.6/site-packages/_pyjq.cpython-36m-x86_64-linux-gnu.so(+0x71e5)[0x7fcdef1671e5]
/lib64/libpython3.6m.so.1.0(+0xf71e2)[0x7fcdf6d771e2]
/lib64/libpython3.6m.so.1.0(_PyObject_FastCallDict+0x90)[0x7fcdf6d28160]
/lib64/libpython3.6m.so.1.0(_PyObject_FastCallKeywords+0x9c)[0x7fcdf6d287ec]
/lib64/libpython3.6m.so.1.0(+0x15098c)[0x7fcdf6dd098c]
/lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x162b)[0x7fcdf6dc637b]
/lib64/libpython3.6m.so.1.0(+0x14f987)[0x7fcdf6dcf987]
/lib64/libpython3.6m.so.1.0(+0x15067a)[0x7fcdf6dd067a]
/lib64/libpython3.6m.so.1.0(+0x150903)[0x7fcdf6dd0903]
/lib64/libpython3.6m.so.1.0(_PyEval_EvalFrameDefault+0x3a7)[0x7fcdf6dc50f7]
/lib64/libpython3.6m.so.1.0(PyEval_EvalCodeEx+0x24f)[0x7fcdf6dd0c9f]
/lib64/libpython3.6m.so.1.0(PyEval_EvalCode+0x3b)[0x7fcdf6dd17ab]
/lib64/libpython3.6m.so.1.0(+0x1d912e)[0x7fcdf6e5912e]
/lib64/libpython3.6m.so.1.0(PyRun_FileExFlags+0xa4)[0x7fcdf6d06e6f]
/lib64/libpython3.6m.so.1.0(PyRun_SimpleFileExFlags+0x39c)[0x7fcdf6d07241]
/lib64/libpython3.6m.so.1.0(Py_Main+0x652)[0x7fcdf6e5f5c2]
python3(main+0x119)[0x400ab9]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fcdf5faf555]
python3[0x400c40]
======= Memory map: ========
00400000-00401000 r-xp 00000000 fd:00 13544                              /usr/bin/python3.6
00601000-00602000 r--p 00001000 fd:00 13544                              /usr/bin/python3.6
00602000-00603000 rw-p 00002000 fd:00 13544                              /usr/bin/python3.6
01ad4000-01c67000 rw-p 00000000 00:00 0                                  [heap]
7fcde8000000-7fcde8021000 rw-p 00000000 00:00 0 
7fcde8021000-7fcdec000000 ---p 00000000 00:00 0 
7fcdeea7a000-7fcdeea8f000 r-xp 00000000 fd:00 33554508                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fcdeea8f000-7fcdeec8e000 ---p 00015000 fd:00 33554508                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fcdeec8e000-7fcdeec8f000 r--p 00014000 fd:00 33554508                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fcdeec8f000-7fcdeec90000 rw-p 00015000 fd:00 33554508                   /usr/lib64/libgcc_s-4.8.5-20150702.so.1
7fcdeec90000-7fcdeed0b000 r-xp 00000000 fd:00 34676113                   /usr/lib64/libonig.so.5.0.0
7fcdeed0b000-7fcdeef0a000 ---p 0007b000 fd:00 34676113                   /usr/lib64/libonig.so.5.0.0
7fcdeef0a000-7fcdeef0c000 r--p 0007a000 fd:00 34676113                   /usr/lib64/libonig.so.5.0.0
7fcdeef0c000-7fcdeef13000 rw-p 0007c000 fd:00 34676113                   /usr/lib64/libonig.so.5.0.0
7fcdeef13000-7fcdeef5e000 r-xp 00000000 fd:00 34675989                   /usr/lib64/libjq.so.1.0.4
7fcdeef5e000-7fcdef15d000 ---p 0004b000 fd:00 34675989                   /usr/lib64/libjq.so.1.0.4
7fcdef15d000-7fcdef15f000 r--p 0004a000 fd:00 34675989                   /usr/lib64/libjq.so.1.0.4
7fcdef15f000-7fcdef160000 rw-p 0004c000 fd:00 34675989                   /usr/lib64/libjq.so.1.0.4
7fcdef160000-7fcdef16b000 r-xp 00000000 fd:00 1267456                    /usr/lib64/python3.6/site-packages/_pyjq.cpython-36m-x86_64-linux-gnu.so
7fcdef16b000-7fcdef36a000 ---p 0000b000 fd:00 1267456                    /usr/lib64/python3.6/site-packages/_pyjq.cpython-36m-x86_64-linux-gnu.so
7fcdef36a000-7fcdef36b000 r--p 0000a000 fd:00 1267456                    /usr/lib64/python3.6/site-packages/_pyjq.cpython-36m-x86_64-linux-gnu.so
7fcdef36b000-7fcdef36d000 rw-p 0000b000 fd:00 1267456                    /usr/lib64/python3.6/site-packages/_pyjq.cpython-36m-x86_64-linux-gnu.so
7fcdef36d000-7fcdef376000 r-xp 00000000 fd:00 33770305                   /usr/lib64/python3.6/lib-dynload/_struct.cpython-36m-x86_64-linux-gnu.so
7fcdef376000-7fcdef576000 ---p 00009000 fd:00 33770305                   /usr/lib64/python3.6/lib-dynload/_struct.cpython-36m-x86_64-linux-gnu.so
7fcdef576000-7fcdef577000 r--p 00009000 fd:00 33770305                   /usr/lib64/python3.6/lib-dynload/_struct.cpython-36m-x86_64-linux-gnu.so
7fcdef577000-7fcdef579000 rw-p 0000a000 fd:00 33770305                   /usr/lib64/python3.6/lib-dynload/_struct.cpython-36m-x86_64-linux-gnu.so
7fcdef579000-7fcdef5b9000 rw-p 00000000 00:00 0 
7fcdef5b9000-7fcdef5c4000 r-xp 00000000 fd:00 33770288                   /usr/lib64/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fcdef5c4000-7fcdef7c3000 ---p 0000b000 fd:00 33770288                   /usr/lib64/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fcdef7c3000-7fcdef7c4000 r--p 0000a000 fd:00 33770288                   /usr/lib64/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fcdef7c4000-7fcdef7c5000 rw-p 0000b000 fd:00 33770288                   /usr/lib64/python3.6/lib-dynload/_json.cpython-36m-x86_64-linux-gnu.so
7fcdef7c5000-7fcdef805000 rw-p 00000000 00:00 0 
7fcdef805000-7fcdef808000 r-xp 00000000 fd:00 33770287                   /usr/lib64/python3.6/lib-dynload/_heapq.cpython-36m-x86_64-linux-gnu.so
7fcdef808000-7fcdefa07000 ---p 00003000 fd:00 33770287                   /usr/lib64/python3.6/lib-dynload/_heapq.cpython-36m-x86_64-linux-gnu.so
7fcdefa07000-7fcdefa08000 r--p 00002000 fd:00 33770287                   /usr/lib64/python3.6/lib-dynload/_heapq.cpython-36m-x86_64-linux-gnu.so
7fcdefa08000-7fcdefa0a000 rw-p 00003000 fd:00 33770287                   /usr/lib64/python3.6/lib-dynload/_heapq.cpython-36m-x86_64-linux-gnu.so
7fcdefa0a000-7fcdefa4a000 rw-p 00000000 00:00 0 
7fcdefa4a000-7fcdf5f8d000 r--p 00000000 fd:00 33747196                   /usr/lib/locale/locale-archive
7fcdf5f8d000-7fcdf6150000 r-xp 00000000 fd:00 33592045                   /usr/lib64/libc-2.17.so
7fcdf6150000-7fcdf6350000 ---p 001c3000 fd:00 33592045                   /usr/lib64/libc-2.17.soAborted

I wrote a workaround in my wrapper class that prepends jq as statements to the script before compilation, so this isn't a show-stopper for me.

doloopwhile commented 3 years ago

It is not reproduced on my latest version.