search

earwig / mwparserfromhell

A Python parser for MediaWiki wikicode
https://mwparserfromhell.readthedocs.io/
MIT License
758 stars 75 forks source link

mwparserfromhell fails with Python 3.13.0-rc.1 #327

Open xqt opened 3 months ago

xqt commented 3 months ago

The exception occurs when importing mwparserfromhell. See here for a failing example. BTW it works with Python 3.14. Here is the traceback:

SystemError: Objects/dictobject.c:3774: bad argument to internal function

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/runner/work/pywikibot/pywikibot/pwb.py", line 40, in <module>
    sys.exit(main())
             ~~~~^^
  File "/home/runner/work/pywikibot/pywikibot/pwb.py", line 36, in main
    runpy.run_path(str(path), run_name='__main__')
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen runpy>", line 2[87](https://github.com/wikimedia/pywikibot/actions/runs/10318192693/job/28564043859#step:7:88), in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line [88](https://github.com/wikimedia/pywikibot/actions/runs/10318192693/job/28564043859#step:7:89), in _run_code
  File "/home/runner/work/pywikibot/pywikibot/pywikibot/scripts/wrapper.py", line 299, in <module>
    import pywikibot as pwb
  File "/home/runner/work/pywikibot/pywikibot/pywikibot/__init__.py", line 60, in <module>
    from pywikibot.site import BaseSite as _BaseSite
  File "/home/runner/work/pywikibot/pywikibot/pywikibot/site/__init__.py", line 13, in <module>
    from pywikibot.site._apisite import APISite
  File "/home/runner/work/pywikibot/pywikibot/pywikibot/site/_apisite.py", line 25, in <module>
    from pywikibot.data import api
  File "/home/runner/work/pywikibot/pywikibot/pywikibot/data/api/__init__.py", line 26, in <module>
    from pywikibot.data.api._requests import CachedRequest, Request, encode_url
  File "/home/runner/work/pywikibot/pywikibot/pywikibot/data/api/_requests.py", line 41, in <module>
    from pywikibot.textlib import removeDisabledParts, removeHTMLParts
  File "/home/runner/work/pywikibot/pywikibot/pywikibot/textlib.py", line 39, in <module>
    import mwparserfromhell as wikitextparser
  File "/opt/hostedtoolcache/Python/3.13.0-rc.1/x64/lib/python3.13/site-packages/mwparserfromhell/__init__.py", line 33, in <module>
    from . import definitions, nodes, parser, smart_list, string_mixin, utils, wikicode
  File "/opt/hostedtoolcache/Python/3.13.0-rc.1/x64/lib/python3.13/site-packages/mwparserfromhell/parser/__init__.py", line 31, in <module>
    from ._tokenizer import CTokenizer
  File "<frozen importlib._bootstrap>", line 469, in _lock_unlock_module
  File "<frozen importlib._bootstrap>", line 432, in _get_module_lock
SystemError: <built-in function acquire_lock> returned a result with an exception set
legoktm commented 3 months ago

I was able to get the stacktrace by installing the python3.13-debug Fedora package, and then installing the package in a python3.13d-powered virtualenv and running it under gdb:

$ gdb python
GNU gdb (Fedora Linux) 14.2-3.fc40
(gdb) run -c "import mwparserfromhell"                                                                                                                                                                                                        
Starting program: /home/user/github/earwig/mwparserfromhell/.venv/bin/python -c "import mwparserfromhell"
Downloading separate debug info for /lib64/libpython3.13d.so.1.0                                                                                                                                                                              
[Thread debugging using libthread_db enabled]                                                                                                                                                                                                 
Using host libthread_db library "/lib64/libthread_db.so.1".
python: /builddir/build/BUILD/Python-3.13.0rc1/Python/ceval.c:769: _PyEval_EvalFrameDefault: Assertion `!_PyErr_Occurred(tstate)' failed.                                                                                                     

Program received signal SIGABRT, Aborted.
__pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
Downloading source file /usr/src/debug/glibc-2.39-22.fc40.x86_64/nptl/pthread_kill.c
44            return INTERNAL_SYSCALL_ERROR_P (ret) ? INTERNAL_SYSCALL_ERRNO (ret) : 0;                                                                                                                                                       
(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007ffff74a86d3 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x00007ffff744fc4e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff7437902 in __GI_abort () at abort.c:79
#4  0x00007ffff743781e in __assert_fail_base (fmt=0x7ffff75c3b98 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n", assertion=assertion@entry=0x7ffff7af3e33 "!_PyErr_Occurred(tstate)", 
    file=file@entry=0x7ffff7c37708 "/builddir/build/BUILD/Python-3.13.0rc1/Python/ceval.c", line=line@entry=769, function=function@entry=0x7ffff7b54e20 <__PRETTY_FUNCTION__.71.lto_priv.5> "_PyEval_EvalFrameDefault") at assert.c:94
#5  0x00007ffff7447d87 in __assert_fail (assertion=0x7ffff7af3e33 "!_PyErr_Occurred(tstate)", file=0x7ffff7c37708 "/builddir/build/BUILD/Python-3.13.0rc1/Python/ceval.c", line=769, 
    function=0x7ffff7b54e20 <__PRETTY_FUNCTION__.71.lto_priv.5> "_PyEval_EvalFrameDefault") at assert.c:103
#6  0x00007ffff78e21a0 in _PyEval_EvalFrameDefault (tstate=0x7ffff7db5110 <_PyRuntime+298832>, frame=0x7ffff7fb7c08, throwflag=0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Python/ceval.c:769
#7  0x00007ffff78e006f in _PyEval_EvalFrame (tstate=0x7ffff7db5110 <_PyRuntime+298832>, frame=0x7ffff7fb7c08, throwflag=0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Include/internal/pycore_ceval.h:119
#8  0x00007ffff7904a96 in _PyEval_Vector (tstate=0x7ffff7db5110 <_PyRuntime+298832>, func=0x7ffff736a990, locals=0x0, args=0x7ffffffeee68, argcount=1, kwnames=0x0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Python/ceval.c:1806
#9  0x00007ffff772aaee in _PyFunction_Vectorcall (func=0x7ffff736a990, stack=0x7ffffffeee68, nargsf=9223372036854775809, kwnames=0x0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Objects/call.c:413
#10 0x00007ffff7729873 in _PyObject_VectorcallTstate (tstate=0x7ffff7db5110 <_PyRuntime+298832>, callable=0x7ffff736a990, args=0x7ffffffeee68, nargsf=9223372036854775809, kwnames=0x0)
    at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Include/internal/pycore_call.h:168
#11 0x00007ffff772bfcf in PyObject_VectorcallMethod (name=0x7ffff7d7b618 <_PyRuntime+62552>, args=0x7ffffffeee68, nargsf=9223372036854775809, kwnames=0x0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Objects/call.c:856
#12 0x00007ffff796210c in PyObject_CallMethodOneArg (self=0x7ffff73d4dd0, name=0x7ffff7d7b618 <_PyRuntime+62552>, arg=0x7ffff6eab5e0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Include/cpython/abstract.h:74
#13 0x00007ffff79631ed in import_ensure_initialized (interp=0x7ffff7d858c0 <_PyRuntime+104192>, mod=0x7ffff72ab470, name=0x7ffff6eab5e0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Python/import.c:228
#14 0x00007ffff796a6f0 in PyImport_ImportModuleLevelObject (name=0x7ffff6eab5e0, globals=0x7ffff73d49b0, locals=0x0, fromlist=0x7ffff708d4f0, level=0) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Python/import.c:3751
#15 0x00007ffff796aa69 in PyImport_ImportModuleLevel (name=0x7ffff7e47e5a "mwparserfromhell", globals=0x7ffff73d49b0, locals=0x0, fromlist=0x7ffff708d4f0, level=0)
    at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Python/import.c:3852
#16 0x00007ffff7e460db in load_defs () at src/mwparserfromhell/parser/ctokenizer/tokenizer.c:272
#17 0x00007ffff7e46375 in PyInit__tokenizer () at src/mwparserfromhell/parser/ctokenizer/tokenizer.c:324
#18 0x00007ffff796d3fb in _PyImport_RunModInitFunc (p0=0x7ffff7e46280 <PyInit__tokenizer>, info=0x7ffffffef230, p_res=0x7ffffffef190) at /usr/src/debug/python3.13-3.13.0~rc1-1.fc40.x86_64/Python/importdl.c:423
#19 0x00007ffff796662f in import_run_extension (tstate=0x7ffff7db5110 <_PyRuntime+298832>, p0=0x7ffff7e46280 <PyInit__tokenizer>, info=0x7ffffffef230, spec=0x7ffff70269a0, modules=0x7ffff73b2390)

After reading https://docs.python.org/3.13/whatsnew/3.13.html#c-api-changes I suspected this was related to:

And indeed, swapping in the new PyEval_GetFrameGlobals/PyEval_GetFrameLocals fixes this issue, but ends up triggering other issues related to entitydefs that I didn't figure out.

xqt commented 2 months ago

Seems it works again with Python 3.13.0rc2