How to analyze code with mpi?

[frankroeder]

Hello,

recently I've tried to analyze code containing mpi with scan-build. If I pass mpicc to --use-cc it won't work. For the already named example of https://github.com/0ax1/MPI-Checker/tree/master/examples there are two possibility:

• In case of CMake it is required to generate the compilation database with Cmake itself and invoke analyze-build afterwards.

• In case of make, its required to set environmental variables like export LDFLAGS=mpicc --showme:link, export CFLAGS=mpicc --showme:compile and export CXXFLAGS=mpicc --showme:compile, before scan-build can be invoked with --use-cc=clang. Otherwise he won't be able the realize "mpi.h".

[rizsotto]

I don't have much experience with MPI. But as I understood from the @0ax1 description, in case of make you can still generate the compilation database with intercept-build and then run analyze-build... As a rule of thumb, I always recommend to run intercept-build and analyze-build instead of scan-build... But give me some time, next week will try to make some experiment with it.

[0ax1]

Currently, both scan-build and intercept-build do not work. intercept-build simply produces an empty compile_commands.json whereas scan-build gives an empty report . When glancing at compilation.py my suspicion is that it might be because the MPI compilers (mpicc, mpic++ etc.) are treated as regular compilers but not as compiler wrappers which is what they are. In particular, they also implicitly set MPI related link and include flags which is why using --use-cc or --use-c++ might lead to headers or symbols not being found.

[0ax1]

Thanks for having a look at this! 👍

[rizsotto]

Hey Guys, was playing with this. Not sure I've fixed it. Need your feedback. The change is on fix branch. (Did not merged to master yet.) The intercept-build produces a non empty compilation database for MPI compilers too. (Was running against @0ax1 example project.)

[0ax1]

Hey László, thanks again for looking into this. I just gave the fix branch a try but somehow intercept-build still produces an empty compilation database. Which MPI version did you use? I tested it with MPICH 3.2 and Open MPI 2.1.1 . Did you add any flags to intercept-build make?

[rizsotto]

Thanks @0ax1 , was testing with MPICH 3.2 on Fedora 25. No extra flags. Used your repo examples. Will look at it again and try with OPENMPI too.

[rizsotto]

Sorry to be late with this... I did re-check with OpenMPI and MPICH on Fedora 26. Both cases it was generate a non empty compilation database, with the expanded flags from mpicc -show command. This is released as 2.0.8 on master.

[0ax1]

No problem. Great to hear that it is working with multiple MPI implementations. I was wondering why it still wasn't working on my current macOS setup but then remembered from your libEAR page that it is necessary to disable system integrity protection on macOS in order to produce a compilation database. So after disabling system integrity protection, intercept-build also works on macOS. I think it would be great if this information would be in the README of this repo. What's strange is that scan-build still doesn't produce a report (scan-build: Removing directory ... because it contains no report.). Do you have an idea why this is the case?

One thing I noticed about 5af4396 is that the open_mpi_compiler test was commented out instead of removed: https://github.com/rizsotto/scan-build/commit/5af43962e94cd398fe5919053760d0479460fa43#diff-e1dbf5104a4e733ff3e792a2b25272cbR54. Was that intentional?

Thanks again for making this work!

[rizsotto]

@0ax1 thanks for your comment! You touched multiple issues, I hope I can answer them all. ;)

Because now the mpi compiler wrappers generates random flags (in a sense that it's not deterministic for every installation), it is hard to write a unit test which asserts expected flags. I might need to rework that for proper dependency injection, to make it properly testable. Until then it is commented out.

The README file talks about different behavior of intercept-build depends on the OS security protection. When dynamic linker pre-load is not available it falls back to compiler wrappers. The wrapper solution is sensitive about the build system. The build system shall use CC and CXX environment variable to call the compiler. If it does intercept-build will work as expected and be able to build the compilation database. (Same story about the scan-build.)

[0ax1]

I think you did :). Good to know how these aspects make a difference.

[Raamkrishnan]

Sorry to be late with this... I did re-check with OpenMPI and MPICH on Fedora 26. Both cases it was generate a non empty compilation database, with the expanded flags from mpicc -show command. This is released as 2.0.8 on master.

Hi,

I am using intercept-build to generate the compilation data base. Our code use OpenMPI. I use the following line to generate the .JSON file:

intercept-build --override-compiler --use-cc=mpicc --use-c++=mpicxx $intercept_make

JSON file created. But the created file is empty. Could you tell me how to use the intercept-build with OpenMPI?

[rizsotto]

Hey @Raamkrishnan , the command you've used looks good to me. But the devil is in the details... Feel free to create another ticket.

Which OS do you use? Which version of this tool you are using? How did you installed?

The other relevant information can be, how is your project looks like? What build system it's using? Does the build respect the CC and CXX environment variables for picking the compiler?

You can also run the same command with an extra -vvvv append to the flags. This will make intercept-build write verbose logs, which you can attach to the ticket.

[Raamkrishnan]

Thank you for your reply. I use Ubuntu 18.04.2 LTS. I am using scan-build-2.0.16 and I downloaded that using pip3 (python3.6). I use openmpi-1.10.7.

I want to generate the compilation data base for IBAMR code (open source code to simulate fluid flows). Then I will read this database in some editors so that I can debug the code. This project build system is based on GNU Make. This project relies upon several other open-source libraries such as PETSc, SAMRAI, LibMesh.

I am just trying to generate compilation database for PETSc now. I used the following commands:

**intercept_make="make"

if [ $SETUP_PETSC -eq 1 ]; then echo "setting up petsc" cd $PETSC_DIR echo "working directory is: $PWD" echo "PETSC_ARCH is: $PETSC_ARCH" make PETSC_ARCH=$PETSC_ARCH clean rm $PWD/compile_commands.json intercept-build --override-compiler --use-cc=mpicc --use-c++=mpicxx $intercept_make PETSC_ARCH=$PETSC_ARCH all compdb -p . list > compile_commands-new.json cp $PWD/compile_commands-new.json $PROJECT_ROOT/petsc-compile_commands.json if [ $REBUILD_LIBRARIES -eq 1 ]; then make PETSC_ARCH=$PETSC_ARCH all fi cd $PROJECT_ROOT fi**

compile_commands.json file is created. but it is empty. Like you suggested, I also ran with the following line:

intercept-build -vvvv --override-compiler --use-cc=mpicc --use-c++=mpicxx $intercept_make

The results is same. I am attaching the output of above lines in the terminal. Please let me know your suggestions

output.TXT

[rizsotto]

Thanks to come back. Was try to read the PETSc make file, but failed to understand the relevant details. :/

What I noticed that you forced the tool to use the "compiler wrapper" to intercept the compilation commands with the --override-compiler flag. But the build not necessarily pick that up. Is there any reason why you want to run like this?

I would recommend instead the intercept-build -vvvv $intercept_make command. Could you try this?

[Raamkrishnan]

Thank you for your suggestion. It works for me. I used this command:

intercept-build --use-cc=mpicc --use-c++=mpicxx $intercept_make

It creates non empty compilation data base. Could you tell me what was happening when I used the --override-compiler flag?

[rizsotto]

I'm glad that it worked... for more details please read the readme file in this project. It has a section to explain the differences between the modes... I think specifying the --use-cc and the --use-cxx is also not necessary. If you are interested to make the command shorter.

[Raamkrishnan]

Yse. --use-cc and --use-cxx are also redundant in this case. I tried it. Just I have to use:

intercept-build $intercept_make

to create the non empty compilation database. Thank you for your suggestions.