search

rizsotto / Bear

Bear is a tool that generates a compilation database for clang tooling.
GNU General Public License v3.0
4.83k stars 313 forks source link

clang-check and clang-tidy broke down with compile_commands.json #550

Closed 2403772980ygy closed 9 months ago

2403772980ygy commented 1 year ago

Describe the bug Whenever I use clang-check and clang-tidy to check my files they errors similar errors:

clang-tidy:

1 error generated.
Error while processing /home/erina/cpp/test.cpp.
error: error reading 'pic': No such file or directory [clang-diagnostic-error]
Found compiler error(s).

clang-check:

error: error reading 'pic': No such file or directory
1 error generated.
Error while processing /home/erina/cpp/test.cpp.

To Reproduce Directory: test test.cpp Makefile

in the makefile:

test: test.cpp
    clang++ -o test test.cpp

remove ./test and run bear -- make

Expected behavior clang-check and clang-tidy should run fine.

Environment:

Additional context

Well, there is a ton to be said......

I guess this is not (completely) bear's fault.

The compile_commands.json looks as follows:

[
  {
    "arguments": [
      "/usr/lib/llvm/17/bin/clang++",
      "-c",
      "-o",
      "test",
      "test.cpp"
    ],
    "directory": "/home/erina/cpp",
    "file": "/home/erina/cpp/test.cpp",
    "output": "/home/erina/cpp/test"
  },
  {
    "arguments": [
      "/usr/lib/llvm/17/bin/clang-17",
      "-cc1",
      "-triple",
      "x86_64-pc-linux-gnu",
      // redacted some lines
      "-main-file-name",
      "-mrelocation-model",
      "pic",
      "-pic-level",
      "2",
      "-pic-is-pie",
      "-mframe-pointer=all",
      "-fmath-errno",
      "-ffp-contract=on",
      "-fno-rounding-math",
      // redacted some lines
      "-D__GCC_HAVE_DWARF2_CFI_ASM=1",
      "-x",
      "c++",
      "-o",
      "/tmp/test-80cc08.o",
      "test.cpp"
    ],
    "directory": "/home/erina/cpp",
    "file": "/home/erina/cpp/test.cpp",
    "output": "/tmp/test-80cc08.o"
  }
]

As you can see: the -main-file-name should be test.cpp, but after it comes -mrelocation-model, which is intepreted as the supplement to -main-file-name, and pic is intepreted as a file.

The output of clang++ -o test test.cpp --verbose: (this have the test.cpp after -main-file-name)

clang version 17.0.1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/lib/llvm/17/bin
Configuration file: /etc/clang/x86_64-pc-linux-gnu-clang++.cfg
System configuration file directory: /etc/clang
Selected GCC installation: /usr/lib/gcc/x86_64-pc-linux-gnu/13
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Selected multilib: .;@m64
Found CUDA installation: /opt/cuda, version 
 "/usr/lib/llvm/17/bin/clang-17" -cc1 -triple x86_64-pc-linux-gnu -emit-obj -mrelax-all -dumpdir test- -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/erina/cpp -resource-dir /usr/lib/llvm/17/bin/../../../../lib/clang/17 -include /usr/include/gentoo/fortify.h -include /usr/include/gentoo/maybe-stddefs.h -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13 -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/x86_64-pc-linux-gnu -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/backward -internal-isystem /usr/lib/llvm/17/bin/../../../../lib/clang/17/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/include -internal-externc-isystem /include -internal-externc-isystem /usr/include -fdeprecated-macro -fdebug-compilation-dir=/home/erina/cpp -ferror-limit 19 -stack-protector 2 -fstack-clash-protection -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/test-ddf043.o -x c++ test.cpp
clang -cc1 version 17.0.1 based upon LLVM 17.0.1 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "/usr/local/include"
ignoring nonexistent directory "/usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/include"
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13
 /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/x86_64-pc-linux-gnu
 /usr/lib/gcc/x86_64-pc-linux-gnu/13/include/g++-v13/backward
 /usr/lib/llvm/17/bin/../../../../lib/clang/17/include
 /usr/include
End of search list.
 "/usr/bin/x86_64-pc-linux-gnu-ld.bfd" -pie --hash-style=gnu --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o test /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../lib64/Scrt1.o /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../lib64/crti.o /usr/lib/gcc/x86_64-pc-linux-gnu/13/crtbeginS.o -L/usr/lib/gcc/x86_64-pc-linux-gnu/13 -L/usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../lib64 -L/lib/../lib64 -L/usr/lib/../lib64 -L/usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../x86_64-pc-linux-gnu/lib -L/lib -L/usr/lib -z relro /tmp/test-ddf043.o -lstdc++ -lm -lgcc_s -lgcc -lc -lgcc_s -lgcc /usr/lib/gcc/x86_64-pc-linux-gnu/13/crtendS.o /usr/lib/gcc/x86_64-pc-linux-gnu/13/../../../../lib64/crtn.o

I guess this is why bear generated a broken compile_command.json.

Before you send...

2403772980ygy commented 1 year ago

I have to note that adding "test.cpp" after "-main-file-name" in compile_commands.json can make clang-check and clang-tidy run fine.

rizsotto commented 1 year ago

Good catch @2403772980ygy , thanks for the report.

I think the output should only contain the first entry. (The second entry comes from the fact, that clang++ is calling clang-17.)

A quick workaround to get that, if you craft a configuration file, and the output.content.duplicate_filter_fields is set to file. This should result a single entry. (See the man page about the configuration file.)

2403772980ygy commented 12 months ago

Oh, I know why the workaround works. It should be the same without the second entry really.

Honestly, I am kinda curious is the removal of the "test.cpp" line in the compile_commands.json a deliberate act? I guess bear captured the call just right. It just omitted the line in the output.

Also, I'm really wondering why there is a option called -main-file-name. Is that used to sepatate the file from, like, example.o?

rizsotto commented 12 months ago

The second entry (which starts with clang-17 -cc1 ...) is an internal call of the compiler.

Compilers like Clang or Gcc are working like this: you call the compiler executable with the flags that is documented in the manual. This is their "public" interface. And the compiler then call itself with a different set of flags. That is the "private" interface. This is when the real work happens... With this design, the compiler developer has the freedom to change the "private" interface and change the internals, while keep the "public" interface stable.

When we are talking about compilation databases. We only want the "public" interface calls in it.

If you wander what the -main-file-name flag does, check out the compiler internals. (Since this is a flag on the "private" interface.)

AzuxirenLeadGuy commented 9 months ago

A quick workaround to get that, if you craft a configuration file, and the output.content.duplicate_filter_fields is set to file. This should result a single entry.

Hi. This is not working for me. I created a file bear-config.json as 

{
    "compilation": null,
    "output": {
        "content": {
            "include_only_existing_source": true
        },
        "format": {
            "command_as_array": true,
            "drop_output_field": false,
            "duplicate_filter_fields": "file"
        }
    }
}

Then, I use

bear --config .bear-config.json -- clang++ -Wall -Wextra -Wpedantic -g -o ./bin/program.out ./src/program.cpp

It prepares the following file

[
    {
      "arguments": [
        "/usr/llvm/bin/clang++",
        "-c",
        "-Wall",
        "-Wextra",
        "-Wpedantic",
        "-g",
        "-o",
        "./bin/program.out",
        "./src/program.cpp"
      ],
      "directory": "REDACTED_PATH",
      "file": "REDACTED_PATH/src/program.cpp",
      "output": "REDACTED_PATH/bin/program.out"
    },
    {
      "arguments": [
        "/usr/llvm/bin/clang-17",
        "-cc1",
        "-triple",
        "x86_64-unknown-linux-gnu",
        "-emit-obj",
        "-mrelax-all",
        "-dumpdir",
        "./bin/program.out-",
        "-disable-free",
        "-clear-ast-before-backend",
        "-disable-llvm-verifier",
        "-discard-value-names",
        "-main-file-name",
        "-mrelocation-model",
        "pic",
        "-pic-level",
        "2",
        "-pic-is-pie",
        "-mframe-pointer=all",
        "-fmath-errno",
        "-ffp-contract=on",
        "-fno-rounding-math",
        "-mconstructor-aliases",
        "-funwind-tables=2",
        "-target-cpu",
        "x86-64",
        "-tune-cpu",
        "generic",
        "-debug-info-kind=constructor",
        "-dwarf-version=5",
        "-debugger-tuning=gdb",
        "-fcoverage-compilation-dir=REDACTED_PATH",
        "-resource-dir",
        "/usr/llvm/lib/clang/17",
        "-internal-isystem",
        "/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12",
        "-internal-isystem",
        "/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/x86_64-linux-gnu/c++/12",
        "-internal-isystem",
        "/usr/lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/backward",
        "-internal-isystem",
        "/usr/llvm/lib/clang/17/include",
        "-internal-isystem",
        "/usr/local/include",
        "-internal-isystem",
        "/usr/lib/gcc/x86_64-linux-gnu/12/../../../../x86_64-linux-gnu/include",
        "-internal-externc-isystem",
        "/usr/include/x86_64-linux-gnu",
        "-internal-externc-isystem",
        "/include",
        "-internal-externc-isystem",
        "/usr/include",
        "-Wall",
        "-Wextra",
        "-Wpedantic",
        "-fdeprecated-macro",
        "-fdebug-compilation-dir=REDACTED_PATH",
        "-ferror-limit",
        "19",
        "-fgnuc-version=4.2.1",
        "-fcxx-exceptions",
        "-fexceptions",
        "-fcolor-diagnostics",
        "-faddrsig",
        "-D__GCC_HAVE_DWARF2_CFI_ASM=1",
        "-x",
        "c++",
        "-o",
        "/tmp/program-4ca0e0.o",
        "./src/program.cpp"
      ],
      "directory": "REDACTED_PATH",
      "file": "REDACTED_PATH/src/program.cpp",
      "output": "/tmp/program-4ca0e0.o"
    }
  ]

As you can see, the file is repeated for clang++ and clang-17. The second part is clashing with my clang-tidy configuration. Is there any other workaround for this?

rizsotto commented 9 months ago

@AzuxirenLeadGuy you did set the duplicate_filter_fields under format and not content.

If this is still not working, try the compilation.compilers_to_exclude flag to set.

An example config when both is applied.

{
  "compilation": {
    "compilers_to_recognize": [],
    "compilers_to_exclude": [
      "/usr/llvm/bin/clang-17"
    ]
  },
  "output": {
    "content": {
      "include_only_existing_source": true,
      "paths_to_include": [],
      "paths_to_exclude": [],
      "duplicate_filter_fields": "file"
    },
    "format": {
      "command_as_array": true,
      "drop_output_field": false
    }
  }
}