Standard library headers not being found properly bear+gcc under Nix

[varungandhi-src]

Reproduction steps (on Linux):

1. Install Nix
2. Run the following: (borrowed from the official docs)

git clone https://github.com/NixOS/nix.git && cd nix
git checkout 2cdc9c32e746a620458a1dd87655d2a6c3800f64
nix --extra-experimental-features nix-command --extra-experimental-features flakes develop

./bootstrap.sh
./configure $configureFlags --prefix=$(pwd)/outputs/out

sudo apt-get install bear
bear --force-wrapper -- make -j "$NIX_BUILD_CORES"

jq '[.[] | select(.file | contains("legacy.cc"))]' compile_commands.json > min.json
wget https://github.com/sourcegraph/scip-clang/releases/download/v0.2.7/scip-clang-x86_64-linux
chmod +x scip-clang
scip-clang --compdb-path=min.json --show-compiler-diagnostics

In file included from src/libcmd/legacy.cc:1:
src/libcmd/legacy.hh:4:10: fatal error: 'functional' file not found
#include <functional>
         ^~~~~~~~~~~~
1 error generated.

The above error shouldn't come up.

[varungandhi-src]

The functional header is located under /nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/include/c++/11.3.0/functional.

Added the following the the "arguments" array:

      "-isystem",
      "/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/include/c++/11.3.0",

New error:

In file included from src/libcmd/legacy.cc:1:
In file included from src/libcmd/legacy.hh:4:
/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/include/c++/11.3.0/functional:48:10: fatal error: 'bits/c++config.h' file not found
#include <bits/c++config.h>
         ^~~~~~~~~~~~~~~~~~
1 error generated.

Added the following to the "arguments" array:

      "-isystem",
      "/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/include/c++/11.3.0/x86_64-unknown-linux-gnu",

New error:

In file included from src/libcmd/legacy.cc:1:
In file included from src/libcmd/legacy.hh:4:
In file included from /nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/include/c++/11.3.0/functional:48:
In file included from /nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/include/c++/11.3.0/x86_64-unknown-linux-gnu/bits/c++config.h:586:
/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/include/c++/11.3.0/x86_64-unknown-linux-gnu/bits/os_defines.h:39:10: fatal error: 'features.h' file not found
#include <features.h>
         ^~~~~~~~~~~~
1 error generated.

The features.h header is found under /nix/store/4pqv2mwdn88h7xvsm7a5zplrd8sxzvw0-glibc-2.35-163-dev/include/features.h, so add the following to "arguments" after the previous -isystem flags (the after part is important because cstdlib from C++ has #include_next <stdlib.h>):

      "-isystem",
      "/nix/store/4pqv2mwdn88h7xvsm7a5zplrd8sxzvw0-glibc-2.35-163-dev/include",

This removes all compilation errors.

After that, the index's snapshot output contains:

  typedef std::function<void(int, char * *)> MainFunction;
//        ^^^ reference cxx $ std/
//             ^^^^^^^^ reference cxx $ std/function#
//                                           ^^^^^^^^^^^^ definition cxx $ nix/MainFunction#
//                                           documentation No documentation available.

  struct RegisterLegacyCommand
//       ^^^^^^^^^^^^^^^^^^^^^ definition cxx $ nix/RegisterLegacyCommand#
//       documentation No documentation available.
  {
      typedef std::map<std::string, MainFunction> Commands;
//            ^^^ reference cxx $ std/
//                 ^^^ reference cxx $ std/map#
//                     ^^^ reference cxx $ std/
//                          ^^^^^^ reference cxx $ std/string#
//                                  ^^^^^^^^^^^^ reference cxx $ nix/MainFunction#
//                                                ^^^^^^^^ definition cxx $ nix/RegisterLegacyCommand#Commands#
//                                                documentation No documentation available.

We need to figure out how to programatically determine these flags in the general case.

[varungandhi-src]

Full output for gcc -print-search-dirs

install: /nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/
programs: =/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/:/nix/store/mdck89nsfisflwjv6xv8ydj7dj0sj2pn-gcc-11.3.0-lib/lib/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/mdck89nsfisflwjv6xv8ydj7dj0sj2pn-gcc-11.3.0-lib/lib/:/nix/store/dq0xwmsk1g0i2ayg6pb7y87na2knzylh-gcc-wrapper-11.3.0/bin/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/dq0xwmsk1g0i2ayg6pb7y87na2knzylh-gcc-wrapper-11.3.0/bin/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/libexec/gcc/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/libexec/gcc/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/libexec/gcc/x86_64-unknown-linux-gnu/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../../x86_64-unknown-linux-gnu/bin/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../../x86_64-unknown-linux-gnu/bin/
libraries: =/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/4nlgxhb09sdr51nc9hdm8az5b08vzkgx-glibc-2.35-163/lib/:/nix/store/mdck89nsfisflwjv6xv8ydj7dj0sj2pn-gcc-11.3.0-lib/lib/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/mdck89nsfisflwjv6xv8ydj7dj0sj2pn-gcc-11.3.0-lib/lib/:/nix/store/dq0xwmsk1g0i2ayg6pb7y87na2knzylh-gcc-wrapper-11.3.0/bin/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/dq0xwmsk1g0i2ayg6pb7y87na2knzylh-gcc-wrapper-11.3.0/bin/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../../x86_64-unknown-linux-gnu/lib/x86_64-unknown-linux-gnu/11.3.0/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../../x86_64-unknown-linux-gnu/lib/../lib64/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../x86_64-unknown-linux-gnu/11.3.0/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../../lib64/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../../x86_64-unknown-linux-gnu/lib/:/nix/store/1gf2flfqnpqbr1b4p4qz2f72y42bs56r-gcc-11.3.0/lib/gcc/x86_64-unknown-linux-gnu/11.3.0/../../../

[varungandhi-src]

One hacky way to do this programmatically, is that if the compiler is gcc/g++, then take the following file:

// scip-clang-magic-00 -- Locate glibc headers
#include <features.h>
// scip-clang-magic-01 -- Locate platform-specific stdlib headers
#include <bits/c++config.h>
// scip-clang-magic-02 -- Locate platform-independent stdlib headers
#include <cstdlib>

And run it through g++ -C -E tmp.cc, and parse the lines following the magic comments. (-C preserves comments)

This avoids having to detect any platform triples or any extra path calculation, as we get the paths from g++ itself.

I can't think of a better way at the moment.

[varungandhi-src]

The problem reproduces on macOS with Clang too. I had to use Bear 3.1.2 installed separately, the Bear 3.0.20 which was part of the Nix environment didn't work.

For Clang + macOS, the following is enough:

// scip-clang-magic-00 -- Locate libSystem headers
#include <aio.h>
// scip-clang-magic-02 -- Locate platform-independent stdlib headers
#include <cstdlib>

features.h is glibc-specific, so perhaps we should avoid that. aio.h is a POSIX header so it works equally well on Linux with glibc.

bits/c++config.h is not used by Clang on macOS (at least, in some situations), so we can skip that.