facebookarchive / BOLT

Binary Optimization and Layout Tool - A linux command-line utility used for optimizing performance of binaries
2.51k stars 176 forks source link

instr: change assert to allow FD 0 return by __open() #250

Closed undingen closed 2 years ago

undingen commented 2 years ago

In some cases __open() is returning 0 for me. The open syscall will return a negative number (-1) on error so 0 should be valid.

This happens when one compiles a bolt instrumented executable of pyston (python implementation) and afterwards pip installs a package which needs to be compiled and sets CFLAGS=-pipe. I could not reduce it down to a small testcase but I guess it one can trigger when manually closing std{in, out, err}. Every thing seems to work normally when disabling the assert for 0 in getBinaryPath() - I decided to also modify the second case in readDescriptions() even though I did not run into that one yet.

rafaelauler commented 2 years ago

Thanks!