search

MatthieuDartiailh / bytecode

Python module to modify bytecode
https://bytecode.readthedocs.io/
MIT License
302 stars 38 forks source link

Using special optimization to get rid of recursion limitations when compiling really huge code #57

Closed thautwarm closed 4 years ago

thautwarm commented 4 years ago

The problem got raised in some to-python compiler, if you need I can provide more details.

In that case, a generated file sometimes can be larger than 2 MB, with a function containing thousands of basic blocks. I failed at using bytecode even I had done things like sys.setrecursionlimit(5000).

The recursion exception is raised at https://github.com/vstinner/bytecode/blob/df5aa489eff54f04e7f6edb1abf9b448351021ea/bytecode/cfg.py#L89-L99

Currently I workaround it with an evil patch, which modifies the global member bytecode.cfg._compute_stack_size, and finally migrate the use of function stacks to the use of a Python list and make things work.

Could I merge this optimization to bytecode library? At least as an option, to support compiling very large bytecode.

MatthieuDartiailh commented 4 years ago

Closed by #58