Assertion error on !(nregs <= 24)

[mahmoodn]

Hi With the following piece of code

      int nregs = 0;
      CUDA_SAFECALL(
          cuFuncGetAttribute(&nregs, CU_FUNC_ATTRIBUTE_NUM_REGS, p->f));

      std::cout << "nregs = " << nregs << std::endl;
      instrument_function_if_needed(ctx, p->f);
      std::cout << "nregs = " << nregs << std::endl;
      nvbit_enable_instrumented(ctx, p->f, true);   // <= Assertion error

May I know why I receive the following assertion error on exceeding maximum register count?

nregs = 20
<TRACE>
nregs = 20
ASSERT FAIL: function.cpp:774:void Function::gen_new_code(std::unordered_map<std::__cxx11::basic_string<char>, Function*>&): FAIL !(nregs <= 24) MSG: instrumentation function should not use more than 24 registers!

Is that a bug in nvbit?

[x-y-z]

Do you have -maxrregcount=24 when you are compiling your instrumentation function (like compiling the inject_funcs.cu in the example tools)?

[mahmoodn]

Yes. The commands are

nvcc -ccbin=g++ -D_FORCE_INLINES -dc -c -G -g -std=c++11 -I../nvbit_release/core -Xptxas -cloning=no -Xcompiler -Wall -arch=sm_86 -Xcompiler -fPIC tracer_tool.cu -o tracer_tool.o
nvcc -ccbin=g++ -D_FORCE_INLINES -I../nvbit_release/core -maxrregcount=24 -Xptxas -astoolspatch --keep-device-functions -arch=sm_86 -Xcompiler -Wall -Xcompiler -fPIC -c inject_funcs.cu -o inject_funcs.o
nvcc -ccbin=g++ -D_FORCE_INLINES -arch=sm_86 -G -g tracer_tool.o inject_funcs.o -L../nvbit_release/core -lnvbit -L /usr/local/cuda-11.2/lib64 -lcuda -lcudart_static -shared -o tracer_tool.so

[mahmoodn]

I also even tried

      int nregs1 = 0;
      CUDA_SAFECALL(
          cuFuncGetAttribute(&nregs1, CU_FUNC_ATTRIBUTE_NUM_REGS, p->f));
      std::cout << "nregs1 = " << nregs1 << std::endl;

      instrument_function_if_needed(ctx, p->f);

      int nregs2 = 0;
      CUDA_SAFECALL(
          cuFuncGetAttribute(&nregs2, CU_FUNC_ATTRIBUTE_NUM_REGS, p->f));
      std::cout << "nregs2 = " << nregs2 << std::endl;

      nvbit_enable_instrumented(ctx, p->f, true);

But still get the same assertion error on the last line and nreg1=nreg2=20.

[mahmoodn]

It seems that if I drop -G -g in the following make command

nvcc -ccbin=g++ -D_FORCE_INLINES -dc -c -G -g -std=c++11 -I../nvbit_release/core -Xptxas -cloning=no -Xcompiler -Wall -arch=sm_86 -Xcompiler -fPIC tracer_tool.cu -o tracer_tool.o

There assertion error doesn't appear any more.

[x-y-z]

Maybe -G inflated the number of used registers.