earwig / mwparserfromhell

A Python parser for MediaWiki wikicode
https://mwparserfromhell.readthedocs.io/
MIT License
741 stars 74 forks source link

Proper way to deeply copy a Wikicode object #317

Open avidspartan1 opened 6 months ago

avidspartan1 commented 6 months ago

Hi! I'm curious as to the "proper" way to deeply copy a Wikicode object.

So far, I've been using the copy module's deepcopy method successfully, although recently I ran into a recursion error (stacktrace abbreviated):

  ...
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 288, in _reconstruct
    item = deepcopy(item, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 288, in _reconstruct
    item = deepcopy(item, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 172, in deepcopy
    y = _reconstruct(x, memo, *rv)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 271, in _reconstruct
    state = deepcopy(state, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 146, in deepcopy
    y = copier(x, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 231, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "C:\Users\pdhoi\AppData\Local\Programs\Python\Python310\lib\copy.py", line 137, in deepcopy
    d = id(x)
RecursionError: maximum recursion depth exceeded while calling a Python object

One thought was to raise the recursion limit, but I thought I'd post here first to see if there's a better way to deeply copy a Wikicode object.

earwig commented 6 months ago

Hi! I don't have an answer aside from copy.deepcopy. What is the use case, out of curiosity?

The only ways I can see the recursion error happening are if the wikicode is extremely deeply nested (but this shouldn't be possible normally since the parser has a much lower recursion limit than Python) or if a node was somehow made a child of one of its ancestors.

Can you share the code/wikitext?

matkoniecz commented 4 months ago

@avidspartan1 Can you share the code/wikitext triggering this?

(if not I think it can be safely closed)