Segmentation fault when create hashlimit match

ntcong commented 3 years ago

I have this snippet

import iptc

name = 'hashlimit'
value = {'hashlimit-above': '100/sec'}

iptc_match = iptc.ip4tc.Rule().create_match(name)
[iptc_match.set_parameter(k, v) for k, v in value.items()]

which will fail with error Segmentation fault (core dumped) what weird is when I replace the last line from list comprehension to a tuple

(iptc_match.set_parameter(k, v) for k, v in value.items())

it works

my environment: python: 3.7.5 Ubuntu 18.04 python-iptables 1.0.0 iptables v1.6.1

ldx commented 3 years ago

I can reproduce the issue with python 3.7. Looking into the stacktrace:

Program received signal SIGSEGV, Segmentation fault.                                                               [53/866]
0x00000000004d1e7a in PyObject_ClearWeakRefs (object=object@entry=0x7ffff68e8eb8) at ../Objects/weakrefobject.c:899        899     ../Objects/weakrefobject.c: No such file or directory.                                                             (gdb) bt                                                                                                                   #0  0x00000000004d1e7a in PyObject_ClearWeakRefs (object=object@entry=0x7ffff68e8eb8) at ../Objects/weakrefobject.c:899    #1  0x0000000000446991 in func_dealloc (op=op@entry=0x7ffff68e8eb8) at ../Objects/funcobject.c:531                         #2  0x0000000000469a59 in _Py_Dealloc (op=0x7ffff68e8eb8) at ../Objects/object.c:1971
#3  0x00000000004e8e16 in call_function (kwnames=0x0, oparg=<optimized out>, pp_stack=<synthetic pointer>)
    at ../Python/ceval.c:4629
#4  _PyEval_EvalFrameDefault (f=0x928a08, throwflag=<optimized out>) at ../Python/ceval.c:3124
#5  0x00000000004dda6a in PyEval_EvalFrameEx (f=f@entry=0x928a08, throwflag=throwflag@entry=0) at ../Python/ceval.c:547
#6  0x00000000004de488 in _PyEval_EvalCodeWithName (_co=0x7ffff7238400, globals=<optimized out>, locals=<optimized out>,
    args=<optimized out>, argcount=argcount@entry=0, kwnames=0x0, kwargs=0x0, kwcount=0, kwstep=2, defs=0x0, defcount=0,
    kwdefs=0x0, closure=0x0, name=0x0, qualname=0x0) at ../Python/ceval.c:3930
#7  0x00000000004de5f2 in PyEval_EvalCodeEx (_co=_co@entry=0x7ffff7238400, globals=globals@entry=0x7ffff72ca148,
    locals=locals@entry=0x7ffff72ca148, args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0, kwcount=0,
    defs=0x0, defcount=0, kwdefs=0x0, closure=0x0) at ../Python/ceval.c:3959
#8  0x00000000004de620 in PyEval_EvalCode (co=co@entry=0x7ffff7238400, globals=globals@entry=0x7ffff72ca148,
    locals=locals@entry=0x7ffff72ca148) at ../Python/ceval.c:524
#9  0x0000000000519a99 in run_mod (mod=mod@entry=0x96ae60, filename=filename@entry=0x7ffff7202a50,
    globals=globals@entry=0x7ffff72ca148, locals=locals@entry=0x7ffff72ca148, flags=flags@entry=0x7fffffffe41c,
    arena=arena@entry=0x7ffff726a4c0) at ../Python/pythonrun.c:1035
#10 0x000000000051c6ab in PyRun_FileExFlags (fp=fp@entry=0x9541b0,
    filename_str=filename_str@entry=0x7ffff72a1190 "issue318.py", start=start@entry=257,
    globals=globals@entry=0x7ffff72ca148, locals=locals@entry=0x7ffff72ca148, closeit=closeit@entry=1,
    flags=0x7fffffffe41c) at ../Python/pythonrun.c:988
#11 0x000000000051ca10 in PyRun_SimpleFileExFlags (fp=fp@entry=0x9541b0, filename=<optimized out>,
    filename@entry=0x7ffff72a1190 "issue318.py", closeit=closeit@entry=1, flags=flags@entry=0x7fffffffe41c)
    at ../Python/pythonrun.c:429
#12 0x000000000051cbdf in PyRun_AnyFileExFlags (fp=fp@entry=0x9541b0, filename=0x7ffff72a1190 "issue318.py",
    closeit=closeit@entry=1, flags=flags@entry=0x7fffffffe41c) at ../Python/pythonrun.c:84
#13 0x0000000000425daa in pymain_run_file (fp=0x9541b0, filename=0x8d98d0 L"issue318.py", p_cf=p_cf@entry=0x7fffffffe41c)
    at ../Modules/main.c:427
#14 0x0000000000425e6a in pymain_run_filename (pymain=pymain@entry=0x7fffffffe440, cf=cf@entry=0x7fffffffe41c)
    at ../Modules/main.c:1627
#15 0x0000000000425f77 in pymain_run_python (pymain=pymain@entry=0x7fffffffe440) at ../Modules/main.c:2877
#16 0x0000000000429571 in pymain_main (pymain=pymain@entry=0x7fffffffe440) at ../Modules/main.c:3038
#17 0x0000000000429654 in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>) at ../Modules/main.c:3073
#18 0x0000000000422e5b in main (argc=<optimized out>, argv=<optimized out>) at ../Programs/python.c:15

Which is weird - it looks more like a python bug than memory corruption or a segfault in the C lib.

I also tried to repro it with python2.7 and python3.9 - both worked fine. So my conclusion is that we're triggering some CPython bug in some way (your experiment to use a set instead of a list also confirms this).

A few things you can try:

You don't really need the output from set_parameters(), any reason you're using a list comprehension?
Using a for loop instead of the list also avoids the problem.
Can you try upgrading your Python install?

ntcong commented 3 years ago

Will try the latest 3.7 and also 3.8 to see if the issue is there. But since 18.04 is widely used (LTS), should we do something about this?

ldx commented 3 years ago

This is the first time this issue is reported, so I'm unsure how common it is. It's also limited what we can do, unfortunately: I was poking around yesterday to see if tweaking gc might make a difference, but it does not seem so.

I'll keep this on my radar in case someone else also bumps into it. Thanks for reporting it!

ldx / python-iptables

Segmentation fault when create hashlimit match #318