Can't specify the used compiler to Bear

[sar5430]

Describe the bug I'm trying to generate a compilation database for a simple application (cross-compilation from Linux to Windows) using MinGW. I first tried to use bear without any arguments: bear make

which gives the following compile_commands.json

[
    {
        "arguments": [
            "c++",
            "-c",
            "-m32",
            "-static-libstdc++",
            "-static-libgcc",
            "-o",
            "test.exe",
            "test.cpp"
        ],
        "directory": "/tmp/test",
        "file": "test.cpp"
    }
]

for the following Makefile:

CFLAGS=-static -m32
LFLAGS=-static-libstdc++ -static-libgcc
CXX=i686-w64-mingw32-g++

test.exe: test.cpp
    $(CXX) $(CFLAGS) $(LFLAGS) $^ -o $@

clean:
    rm test.exe 

Bear does not seems to recognize i686-w64-mingw32-g++ as a valid compiler and set c++ instead.

Then I tried to use the flag --use-c++ to specify the name of the compiler I use. The command bear --use-c++ i686-w64-mingw32-g++ make gives the exact same compile_commands.json Here's a more verbose output of the command

sar@pc:/tmp/test$ bear -vvvv --use-c++ i686-w64-mingw32-g++ make 
bear: DEBUG: parse_args_for_intercept_build: Raw arguments ['/usr/bin/bear', '-vvvv', '--use-c++', 'i686-w64-mingw32-g++', 'make']
bear: DEBUG: parse_args_for_intercept_build: Parsed arguments: Namespace(append=False, build=['make'], cc='cc', cdb='compile_commands.json', cxx='i686-w64-mingw32-g++', libear='/usr/${LIB}/bear/libear.so', verbose=4)
[...]
i686-w64-mingw32-g++ -static -m32 -static-libstdc++ -static-libgcc test.cpp -o test.exe
bear: DEBUG: run_build: build finished with exit code: 0
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.8qr9nl
bear: DEBUG: _split_command: input was: ['/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/bin/nm', '-n', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/lib/../lib/crt2.o']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.faZt5I
bear: DEBUG: _split_command: input was: ['/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/bin/nm', '-n', '/tmp/cc1b1Fc7.o']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.w6ZJex
bear: DEBUG: _split_command: input was: ['/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/bin/nm', '-n', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/crtbegin.o']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.cAZvlu
bear: DEBUG: _split_command: input was: ['/usr/lib/gcc/i686-w64-mingw32/7.3-win32/collect2', '-m', 'i386pe', '-Bstatic', '-o', 'test.exe', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/lib/../lib/crt2.o', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/crtbegin.o', '-L/usr/lib/gcc/i686-w64-mingw32/7.3-win32', '-L/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/lib/../lib', '-L/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/lib', '/tmp/cc1b1Fc7.o', '-lstdc++', '-lmingw32', '-lgcc', '-lgcc_eh', '-lmoldname', '-lmingwex', '-lmsvcrt', '-ladvapi32', '-lshell32', '-luser32', '-lkernel32', '-lmingw32', '-lgcc', '-lgcc_eh', '-lmoldname', '-lmingwex', '-lmsvcrt', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/crtend.o']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.RiDIx9
bear: DEBUG: _split_command: input was: ['/usr/bin/i686-w64-mingw32-ld', '-m', 'i386pe', '-Bstatic', '-o', 'test.exe', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/lib/../lib/crt2.o', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/crtbegin.o', '-L/usr/lib/gcc/i686-w64-mingw32/7.3-win32', '-L/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/lib/../lib', '-L/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/lib', '/tmp/cc1b1Fc7.o', '-lstdc++', '-lmingw32', '-lgcc', '-lgcc_eh', '-lmoldname', '-lmingwex', '-lmsvcrt', '-ladvapi32', '-lshell32', '-luser32', '-lkernel32', '-lmingw32', '-lgcc', '-lgcc_eh', '-lmoldname', '-lmingwex', '-lmsvcrt', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/crtend.o']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.toyOLi
bear: DEBUG: _split_command: input was: ['/usr/bin/i686-w64-mingw32-as', '--32', '-o', '/tmp/cc1b1Fc7.o', '/tmp/cck6bhFL.s']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.BpSmWU
bear: DEBUG: _split_command: input was: ['/usr/lib/gcc/i686-w64-mingw32/7.3-win32/../../../../i686-w64-mingw32/bin/nm', '-n', '/usr/lib/gcc/i686-w64-mingw32/7.3-win32/crtend.o']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.HNAyDV
bear: DEBUG: _split_command: input was: ['/usr/lib/gcc/i686-w64-mingw32/7.3-win32/cc1plus', '-quiet', '-U_REENTRANT', 'test.cpp', '-quiet', '-dumpbase', 'test.cpp', '-m32', '-mtune=generic', '-march=pentiumpro', '-auxbase', 'test', '-o', '/tmp/cck6bhFL.s']
bear: DEBUG: parse_exec_trace: parse exec trace file: /tmp/intercept-o1zlwrj8/execution.k3GUdI
bear: DEBUG: _split_command: input was: ['i686-w64-mingw32-g++', '-static', '-m32', '-static-libstdc++', '-static-libgcc', 'test.cpp', '-o', 'test.exe']
bear: DEBUG: _split_command: output is: CompilationCommand(compiler='c++', phase=[], flags=['-m32', '-static-libstdc++', '-static-libgcc'], files=['test.cpp'], output=['test.exe'])

Did I miss anything regarding the cross-compilation ?

Thanks.

To Reproduce create the file test.cpp:

#include <windows.h>

int main(int argc, char** argv)
{
    MessageBoxA(0, "Hello", "Hello", 0);
    return 0;
}

and the Makefile from the previous section in the same directory. Install MinGW then use bear make in the directory.

Expected behavior I was expected to see i686-w64-mingw32-g++ instead of c++ in the compile_commands.json

[sar5430]

Oh, I didn't notice that Bear was rewritten in version 3. Let me check with that version.

[rizsotto]

Hey @mastabasta , thanks for the report.

It's a long standing issue with Bear that it replaces the compiler name. The version 2.4.4 would spit the compiler name as you expected. (Get the latest 3.0 might be harder to install because of dependencies.)

[koutheir]

This makes bear useless for any kind of cross-compilation. This is very unfortunate :disappointed:

[rizsotto]

Let me be clear. This issue does not tell me which version you guys are trying... I've looked to the source of 2.4.4 version and I think it solves this issue. I did not try to reproduce the bug myself, but waiting for somebody to confirm that.

@koutheir what is your intent with your comment?

[koutheir]

This issue does not tell me which version you guys are trying...

I used the latest release 3.0.3.

what is your intent with your comment?

Express interest in this issue and provide an affected use case.

[rizsotto]

@koutheir can you attach a verbose log from your attempt? (@0xstan logs are from the 2.x branch) The 3.0 branch does not have --use-cc, --use-cxx and --use-only flags. It captures all execution but does not differentiate the entries by the used compiler, and it does not replace the compiler name.

I am curious how the new version is performing on cross compilation, because I've not tested those cases yet.

[koutheir]

The 3.0 branch does not have --use-cc, --use-cxx and --use-only flags.

I didn't know that, because the documentation still mentions them.

It captures all execution but does not differentiate the entries by the used compiler, and it does not replace the compiler name.

For a complex cross-compilation where both host and target tool chains are used, the flags --use-cc, --use-cxx and --use-only are needed.

I am curious how the new version is performing on cross compilation, because I've not tested those cases yet.

Without the mentioned flags, version 3.0.3 does not support my use case of cross-compilation. I switched back to version 2.4.4, and that version works as expected.

[jbmaillet]

I work 100% on embedded products (Linux, Android, bare metal), on such use cases with a mix of native and cross-compilers (3 different ones, two 64 bits target CPU and a 32 bit one). I have no problem with 3.0.3.

The only change between 2.4.4 and 3.0.3 that had an impact on my workflow is in the generated compilation database: for each compilation unit, with 2.4.4 the "file" attribute was relative to "directory". Whereas with 3.0.3, both are absolute.

[rizsotto]

@jbmaillet I'm happy that it works for you.

Can you give me insights what is more desired output format? Relative vs absolute path in the file attribute? Or relative vs absolute in the directory attribute? What do you use the output for? (Editor support, tooling or something else?)

[jbmaillet]

I do not care if the output format uses relative or absolute paths. If it does not start with a "/", I know it's relative and I have to resolve it using both "file" and "directory", else it's absolute and I don't need to do anything. I changed my workflow/code to adapt to that. I am also fully aware that relatives paths are nice and user friendly (ex relative to the root of a project), but can be a can of worms (relative to what, in which context? define "root of a project"? define "project"?). Absolute paths are dumb, but 100% unambiguous.

As of today, I only use bear to establish which files are compiled and which are not. This may and will evolve in the future with inspection of the compilation flags.

Knowing which files are compiled and which are not is not so obvious for highly configurable software components. Ex: a Linux kernel is about 70,000 files, but depending on the build configuration and the 13,000+ config options, for any actual embedded industrial application (that is to say "not general purpose PC use like vendors/RedHat/Canonical/Ubuntu...") only about 20% of these files will be compiled (I've seen this "20%" metric remarkably stable over 4 years and hundreds of inspections on a dozen products). For ex a system may support USB, Wi-Fi, or Bluetooth, some may not. This knowledge about which files actually end up in the final product is important in my context.