Chia-Network / clvm_tools

Tools for clvm development
Apache License 2.0
51 stars 32 forks source link

Deeply nested clvm program cannot run due to Windows10's poor stack memory #74

Open ChiaMineJP opened 3 years ago

ChiaMineJP commented 3 years ago

Test environment

- -
OS Windows10 Pro
CPU AMD Ryzen9 5900X
MEM 64GB@3200MHz
Python 3.7.8

How to reproduce

# powershell
C:\Users\XXX\PycharmProjects\> git clone https://github.com/Chia-Network/clvm_tools
C:\Users\XXX\PycharmProjects\> cd clvm_tools
C:\Users\XXX\PycharmProjects\clvm_tools> python3 -m venv ven
C:\Users\XXX\PycharmProjects\clvm_tools> . venv\Scripts\activate.bat
C:\Users\XXX\PycharmProjects\clvm_tools> pip install -e .
C:\Users\XXX\PycharmProjects\clvm_tools> pip install clvm_rs
C:\Users\XXX\PycharmProjects\clvm_tools> python costs\generate-benchmark.py
C:\Users\XXX\PycharmProjects\clvm_tools> brun .\test-programs\all\all_nest-1-962.clvm
1
C:\Users\XXX\PycharmProjects\clvm_tools> echo $LastExitCode
0
C:\Users\XXX\PycharmProjects\clvm_tools> brun .\test-programs\all\all_nest-1-1002.clvm
C:\Users\XXX\PycharmProjects\clvm_tools> echo $LastExitCode
-1073741571

Maybe sys.setrecursionlimit(20000) does not increase stack size, and AFIK there are no easy way to increase the size on Windows.

It is highly unlikely that such a deep nested clvm program is used on coin spend, but at least you should put this fact into your brain.

But there is another approach. Replace recursive function call into loop. This can utilize heap memory instead of stack memory. If you're interested, I already wrote such a conversion in my javascript implementation of clvm_tools. Here is my work

In nodejs, it failed to run all_nest-1-1502.clvm due to Maximum call stack size exceeded error. So I really tried hard to convert the recursion into loop in order to reduce stack memory consumed. Fortunately as I develop javascript version of clvm_tools really looking like Python's code, you can also apply my work back into the Python's clvm_tools