Not working in OSX 10.15.7

[jinwik]

Describe the bug A clear and concise description of what the bug is. bear create an empty compile_commands.json

To Reproduce bear --verbose -- make COMPILER_WARNINGS_FATAL=false USE_CLANG=true CC="clang -Wno-deprecated-declarations -Wno-implicit-function-declaration"

Environment:

• OS name: OSX
• OS version: 10.15.7 (19H2)
• OS architecture: x86_64
• Bear version: 3.0.7
• Bear install method: brew install bear clang -v Apple clang version 12.0.0 (clang-1200.0.32.28)

Additional context

• Can you give us a reference to the project that you are running against this tool? https://github.com/AdoptOpenJDK/openjdk-jdk8u.git branch: jdk8u282-b08
• What build tools this project is using? make
• What architecture you are trying to compile for? x86_64
• Could you attach build logs? If you can re-run the command which failed (e.g. bear make) with extra verbose log switches (e.g. bear -vvvv make) and send the output of it, that would help a lot.

• Finished building OpenJDK for target 'default' [07:00:17.415867, ic, 74885] Process wait request: done. [pid: 74902] [07:00:17.415896, ic, 74885] Running command. [Exited with 0] [07:00:17.415911, ic, 74885] Stopping gRPC server. [07:00:17.416317, ic, 74885] succeeded with: 0 [07:00:17.417068, br, 74884] Process wait request: done. [pid: 74885] [07:00:17.417093, br, 74884] Running intercept finished. [Exited with 0] [07:00:17.418934, br, 74884] Process spawned. [pid: 22393, command: [/usr/local/Cellar/bear/3.0.7/bin/citnames, --input, compile_commands.sqlite3, --output, compile_commands.json, --run-checks, --verbose]] [07:00:17.418966, br, 74884] Process wait requested. [pid: 22393] [07:00:17.439216, cs, 22393] citnames: 3.0.7 [07:00:17.439840, cs, 22393] arguments: ["/usr/local/Cellar/bear/3.0.7/bin/citnames", "--input", "compile_commands.sqlite3", "--output", "compile_commands.json", "--run-checks", "--verbose"] [07:00:17.439866, cs, 22393] environment: [...] [07:00:17.446242, cs, 22393] arguments parsed: {program: /usr/local/Cellar/bear/3.0.7/bin/citnames, arguments: [{--input: [compile_commands.sqlite3]}, {--output: [compile_commands.json]}, {--run-checks: []}, {--verbose: []}]} [07:00:17.446627, cs, 22393] Configuration: {"compilation":null,"output":{"content":{"include_only_existing_source":true},"format":{"command_as_array":true,"drop_output_field":false}}} [07:00:17.450103, cs, 22393] [pid: 6956] command: {"arguments":["/usr/bin/nm","-Uj","abstractCompiler.o","accessFlags.o","ad_x86_64.o" ...],"program":"/usr/bin/nm","working_dir":"/tmp/src/AdoptOpenJDK/openjdk-jdk8u/build/macosx-x86_64-normal-server-slowdebug/hotspot/bsd_amd64_compiler2/debug"} [07:00:17.450583, cs, 22393] [pid: 6956] failed: No tools recognize this command. [07:00:17.450829, cs, 22393] compilation entries created. [size: 0] [07:00:17.450835, cs, 22393] compilation entries to output. [size: 0] [07:00:17.451739, cs, 22393] compilation entries written. [size: 0] [07:00:17.451838, cs, 22393] succeeded with: 0 [07:00:17.452507, br, 74884] Process wait request: done. [pid: 22393] [07:00:17.452530, br, 74884] Running citnames finished. [Exited with 0] [07:00:17.452687, br, 74884] succeeded with: 0 [07:00:17.450583, cs, 22393] [pid: 6956] failed: No tools recognize this command. [07:00:17.450829, cs, 22393] compilation entries created. [size: 0] [07:00:17.450835, cs, 22393] compilation entries to output. [size: 0] [07:00:17.451739, cs, 22393] compilation entries written. [size: 0] [07:00:17.451838, cs, 22393] succeeded with: 0 [07:00:17.452507, br, 74884] Process wait request: done. [pid: 22393] [07:00:17.452530, br, 74884] Running citnames finished. [Exited with 0] [07:00:17.452687, br, 74884] succeeded with: 0

Before you send... disable SIP

[rizsotto]

Hi @jinwik , thanks for the report... from the log I see that intercept was only caught the nm -Uj abstractCompiler.o accessFlags.o ad_x86_64.o ... command only. I need the logs before that to see what happened during the build.

It could be that the build system does not respect the CC and CXX variables. Looking at the OpenJDK project quickly, as far as I can judge, the build system requires a configure step before you run make. That's when it decide which compiler it uses. Bear impose compiler wrappers during the build process, which we might want to apply to the configure step too.

I would run these commands

• git clean -fdx it cleans all files which might have been generated. (It's like make clean but stronger.)
• intercept --verbose -- bash configure which will impose the compiler wrappers. (The output of this command might be useful for debug if it still produces empty result.)
• sqlite3 commands.sqlite3 "select count(*) from events;" verify if it was captured compiler calls. (Expecting a number greater than zero.)
• bear --verbose -- make all which will runs the build, and we can hope to see the compiler wrappers to be called.

[jinwik]

intercept --verbose -- ./configure --with-debug-level=slowdebug --enable-debug-symbols --disable-zip-debug-info --with-native-debug-symbols=internal --enable-jfr --with-toolchain-type=clang OBJCOPY=objcopy intercept.log

[jinwik]

sqlite3 commands.sqlite3 "select count(*) from events;" 20

[jinwik]

@rizsotto thx for your patience bear.log cat compile_commands.json []

[rizsotto]

Thanks @jinwik , the intercept.log shows where it goes sideways...

...
checking whether we are using the GNU C compiler... yes
checking whether /usr/bin/clang accepts -g... yes
checking for /usr/bin/clang option to accept ISO C89... none needed
checking for clang++... /usr/bin/clang++
checking resolved symbolic links for CXX... no symlink
configure: Using clang C++ compiler version 12.0.0 [Apple clang version 12.0.0 (clang-1200.0.32.28) Target: x86_64-apple-darwin19.6.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin]
checking whether we are using the GNU C++ compiler... yes
checking whether /usr/bin/clang++ accepts -g... yes
...

It should be /usr/local/Cellar/bear/3.0.8/libexec/wrapper.d/cc and not /usr/bin/clang.

It might be looking for directly to clang since, you've added the --with-toolchain-type=clang switch. Could you try to add a clang and clang++ softlink into the /usr/local/Cellar/bear/3.0.8/libexec/wrapper.d/ directory? (Look at other the other files in that directory as example.) And please re-run the git clean -fdx && intercept --verbose -- ./configure ... commands.

[jinwik]

Thanks, @rizsotto but still failed

cd /usr/local/Cellar/bear/3.0.8/libexec/wrapper.d ln -s ../wrapper clang ln -s ../wrapper clang++

intercept --verbose -- ./configure --with-debug-level=slowdebug --enable-debug-symbols --disable-zip-debug-info --with-native-debug-symbols=internal --enable-jfr --with-toolchain-type=clang OBJCOPY=objcopy

[08:25:57.022838, wr, 73935, ppid: 73934] gRPC call requested: supervise::Supervisor::Resolve [08:25:57.024060, wr, 73935, ppid: 73934] gRPC call [Resolve] finished: false [08:25:57.024411, wr, 73935, ppid: 73934] failed with: gRPC call failed: not recognized wrapper" configure: error: A clang compiler is required. Try setting --with-tools-dir. configure exiting with result code 1 [08:25:57.083245, ic, 73321] Process wait request: done. [pid: 73322] [08:25:57.083265, ic, 73321] Running command. [Exited with 1] [08:25:57.083278, ic, 73321] Stopping gRPC server. [08:25:57.083630, ic, 73321] succeeded with: 1

inter.log

[rizsotto]

Thanks @jinwik , this trace was very helpful. I think I've found were it goes wrong.

It uses the wrapper name with full path (instead of the basename of the executable). Will fix it today evening.

[rizsotto]

Hey @jinwik , can you give it a try with the version I have on the devel branch? I think it has a fix for it.

[jinwik]

Thanks @rizsotto, it works. but clion does not seem to recognize the compile_commads.json generated by bear, but it does correctly by making compile-commands (openjdk16). Any suggestions? Thanks! compile-commands-diff.log

[rizsotto]

Good to hear that change on devel branch fix your issue.

About the CLion problem...

• It seems to be a arguments vs command thing. From the specification, both format is valid and should be supported. (Maybe a ticket to that project would help.) As a workaround you can specify a config file to bear (or citnames) to output the field one way or another. Check man citnames for how the config file looks like and how to pass it to the programs. The command_as_array: false will force to use the command field.
• -isysroot and-iframeworkis missing from Bear output. Will look into that.
• -D__STDC_FORMAT_MACROS, -D__STDC_LIMIT_MACROS, -D__STDC_CONSTANT_MACROS are also missing from Bear output. Could you verify if the compilation output had them? I doubt that Bear throws those away.

[jinwik]

sry, that was my problem, I should move compile_commads.json to the directory folder. Great job, thanks @rizsotto I think we can close the issue.