search

avast / retdec

RetDec is a retargetable machine-code decompiler based on LLVM.
https://retdec.com/
MIT License
8.04k stars 951 forks source link

Decompilation fails in phase "signed/unsigned types fixing" due to insufficient stack space #573

Open TheElementalOfDestruction opened 5 years ago

TheElementalOfDestruction commented 5 years ago

I've sent an email to s3rvac@gmail.com with a google drive link to the input file, the output files, and the file that contains the output of running the decompiler. Subject line is the same as this issue

s3rvac commented 5 years ago

Thank you for the report. From the output, retdec-llvmir2hll has failed in phase signed/unsigned types fixing. This might suggest that the reason is insufficient stack space (see #47, #64, #454, #465). I will try to reproduce the issue to confirm.

s3rvac commented 5 years ago

I was able to reproduce the issue. It is indeed caused by insufficient stack space. If you are on Linux, you can increase the maximal stack size via ulimit -Ss X, where X is the new size (you can also use ulimit -Ss unlimited to remove the limit entirely). After removing the stack limit, I was able to get through that phase.

I will leave the issue open for further future investigation why so much stack space is needed.

TheElementalOfDestruction commented 5 years ago

@s3rvac is there anything I can do for this on windows?

s3rvac commented 5 years ago

@TheElementalOfCreation On Windows, the stack size for an executable program is stored in the program. To change the size, you will have to edit the program. You can try using editbin.