psutil segfaults on import (when running pyperf system tune)

[mdboom]

Crash report

It's understandable that some extension modules would break on this fork, but this one is critical to get good, reproducible benchmarking results.

Segfault in psutil when running python -m pyperf system tune.

I tested this with commit d595911.

./python -m venv venv
venv/bin/python -m pip install --no-binary :all: pyperf  # Crashes with pre-built or source-built wheel
venv/bin/python -m pyperf system tune

Error messages

Backtrace:

#0  __strcmp_evex () at ../sysdeps/x86_64/multiarch/strcmp-evex.S:291
#1  0x00005555557206f3 in _PyModule_CreateInitialized (module=module@entry=0x7ffff7b0b220 <moduledef>,
    module_api_version=module_api_version@entry=1013) at Objects/moduleobject.c:249
#2  0x0000555555720b41 in PyModule_Create2 (module=module@entry=0x7ffff7b0b220 <moduledef>,
    module_api_version=module_api_version@entry=1013) at Objects/moduleobject.c:216
#3  0x00007ffff7b07602 in PyInit__psutil_linux () at psutil/_psutil_linux.c:550
#4  0x000055555582e0db in _PyImport_LoadDynamicModuleWithSpec (spec=spec@entry=0x343144357d0, fp=fp@entry=0x0)
    at ./Python/importdl.c:170                                                                                          #5  0x00005555558298f5 in _imp_create_dynamic_impl (module=<optimized out>, file=<optimized out>, spec=0x343144357d0)
    at Python/import.c:2406                                                                                             #6  _imp_create_dynamic (module=<optimized out>, args=<optimized out>, nargs=<optimized out>)
    at Python/clinic/import.c.h:534                                                                                     #7  0x000055555571de31 in cfunction_vectorcall_FASTCALL (func=0x343142d42d0, args=0x343149ce218,
    nargsf=<optimized out>, kwnames=<optimized out>) at Objects/methodobject.c:422
#8  0x0000555555656907 in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=0x7ffff7fb9a20,
    throwflag=<optimized out>) at Python/generated_cases.c.h:3562
#9  0x00005555556bb3cf in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=2, args=0x7fffffffa7b0,
    callable=0x343143e32e0, tstate=0x555555bb5078 <_PyRuntime+374200>) at ./Include/internal/pycore_call.h:92
#10 object_vacall (tstate=tstate@entry=0x555555bb5078 <_PyRuntime+374200>, base=base@entry=0x0,
    callable=0x343143e32e0, vargs=vargs@entry=0x7fffffffa838) at Objects/call.c:828                                     #11 0x00005555556bd572 in PyObject_CallMethodObjArgs (obj=0x0, name=<optimized out>) at Objects/call.c:889
#12 0x000055555582c5e2 in import_find_and_load (abs_name=0x34314bfc390, tstate=0x555555bb5078 <_PyRuntime+374200>)
    at Python/import.c:1759                                                                                             #13 PyImport_ImportModuleLevelObject (name=name@entry=0x34314bfc390, globals=globals@entry=0x0,
    locals=locals@entry=0x0, fromlist=fromlist@entry=0x0, level=0) at Python/import.c:1859
#14 0x00005555557e798e in builtin___import___impl (level=<optimized out>, fromlist=0x0, locals=0x0, globals=0x0,
    name=0x34314bfc390, module=<optimized out>) at Python/bltinmodule.c:274
#15 builtin___import__ (module=<optimized out>, args=<optimized out>, nargs=<optimized out>, kwnames=<optimized out>)
    at Python/clinic/bltinmodule.c.h:106                                                  

Your environment

• Operating system and architecture:

Ubuntu 20.04

[colesbury]

Thanks for the bug report. Can you check that you don't have left over extensions installed from a build of upstream CPython? I've run into that crash when I have psutil-5.9.5-cp36-abi3-... installed. That binary is not ABI compatible and pip will properly exclude it (different ABI flags), but the runtime import mechanism will still try to load it. (And crash because the PyModuleDef layout is different than expected.)

I ran the steps above without a crash.

[mdboom]

Indeed, that was it. Thanks!