When the runtime environment variable CRAY_ACC_DEBUG is set to 1, 2, or 3, CCE writes runtime commentary of accelerator activity to STDERR for debugging purposes; every accelerator action on every PE generates output prefixed with “ACC:”. This may produce a large volume of output and it may be difficult to associate messages with certain routines and/or certain PEs.
With this set of API calls, the programmer can enable or disable output at certain points in the code, and modify the string that is used as the debug message prefix.
Contributors do not currently know how to debug OpenACC code that they add. We should add documentation for this.
This includes environment variables, flags, etc.
For nvhpc + NVIDIA GPUs we have, to start:
from here (https://docs.nvidia.com/hpc-sdk/compilers/openacc-gs/index.html#env-vars)
or
(see here https://indico.euro-fusion.org/event/460/attachments/520/1092/04_-_OpenACC_Debug.pdf)
etc. etc.
For AMD GPUs we have other considerations that @anandrdbz and @wilfonba know about, like
(more info. here https://www.olcf.ornl.gov/wp-content/uploads/2021/04/2021-05-20-Frontier-Tutorial-CCE.pdf)
From here https://cpe.ext.hpe.com/docs/cce/man7/intro_openacc.7.html :