search

NVlabs / NVBit

220 stars 20 forks source link

Instrumentation function register usage limitation #42

Closed zz-Malfurion closed 3 years ago

zz-Malfurion commented 3 years ago

Hi,

I met this error when I tried to inject a big device function.

ASSERT FAIL: function.cpp:774:void Function::gen_new_code(std::unordered_map<std::__cxx11::basic_string<char>, Function*>&): FAIL !(nregs <= 24) MSG: instrumentation function should not use more than 24 registers!

What is the concern of the register usage? The only reason I can imagine is that the instrumentation function might affect the occupancy of the origin kernel.

Can we relax this restriction? Maybe replace it with a warning?

ovilla commented 3 years ago

The injection functions must to be specifically compiled with this line:

inject_funcs.o: inject_funcs.cu
        $(NVCC) $(INCLUDES) -maxrregcount=24 -Xptxas -astoolspatch --keep-device-functions -arch=sm_$(ARCH) -Xcompiler -Wall -Xcompiler -fPIC -c $< -o $@

They have to use at most 24 registers, it is not optional. We should definitely replace the ASSERT with a clear error and exit.