Clangd (Visual Studio Code) Include Path Resolution for QNX Cross Compilation

[ngunderson]

Hello,

Please let me know if I show open this issue or ask this question elsewhere.

I am trying to use the Clangd extension with Visual Studio Code.

I've created a compile_commands.json symbolic link to the appropriate build directory and much of the completion and include path resolution seems to work.

However, the issue is that standard headers (say like #include <iostream>), aren't resolving to the cross platform toolchain I am using (in my case QNX). My compile_commands.json does contain an -isysroot pointing to the QNX installation, allowing me to build, however it doesn't appear that clangd is using this to determine system paths. Is clangd not using this -isysroot or does it not have a way to determine the implicit include paths that are used by the underlying qcc compiler?

Further context, I am using CMake and the Visual Studio Code plugins for CMake and CMake Tools (not Microsoft C++ extension).

Is there anything I am obviously doing wrong? Or are cross compilation includes not supported by cland?

[kadircet]

normally clangd should respect the isysroot options being passed, you can verify those by looking at the clangd logs. there should be a line that looks like: updating $FILE_NAME$ with command $COMMANDS$.

But sometimes custom toolchains have other implicit header search paths. Clangd can extract those from the toolchain's driver if you allow the execution with --query-driver=/path/to/your/compiler*". The path is usually specified as the first argument in your compile commands.json, so you can check what it looks like. Also thein the--query-driverwas intentional. For example if there are some files that are being compiled with gcc, and some with g++ you can just sayg` to allow execution for both.

If these do not work, please provide clangd logs, preferably by passing -log=verbose to clangd. See http://clangd.llvm.org/troubleshooting.html#gathering-logs for details.

[diggit]

Hi, I have same issue. My setup is vscode + cmake and arm-none-eabi gcc toolchain. Standard headers are not found.

Here is one record from compile_commands.json

{
    "directory": "/home/diggit/dev/test/build",
    "command": "/usr/bin/arm-none-eabi-g++ -DBUILD=\"\" -DDEVICE=STM32F302 -DGIT_VERSION_SHORT=\"\" -DHWVERSION=0 -DSTM32F302x8 -DUSE_FULL_LL_DRIVER -DUSE_HAL_DRIVER -D__packed=\"__attribute__((__packed__))\" -D__weak=\"__attribute__((weak))\" -Dtimegm=mktime -I../Inc -I../Src -isystem ../Drivers/CMSIS/Include -isystem ../Drivers/CMSIS/Device/ST/STM32F3xx/Include -isystem ../Drivers/STM32F3xx_HAL_Driver/Inc -ffunction-sections -fdata-sections -Werror=int-in-bool-context  -Werror=switch -Werror=return-type -Werror=parentheses -Wfatal-errors -Wno-psabi -Wno-unused-function  -Wall -Wextra -Wundef -Wnull-dereference -Wcast-align -Wvla -Wmissing-format-attribute -Wuninitialized -Winit-self -Wdouble-promotion -Wstrict-aliasing -Wno-unused-parameter -pedantic -funsigned-char -fdiagnostics-color=always -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables --specs=nano.specs --specs=nosys.specs -fno-use-cxa-atexit -Wint-to-pointer-cast -Wold-style-cast -Weffc++ -Wno-register -fno-rtti -fno-exceptions -fno-threadsafe-statics -flto -mthumb -mcpu=cortex-m4 -mfloat-abi=hard -mfpu=fpv5-sp-d16 -mthumb -mabi=aapcs -Og -g -DDEBUG -DUSE_FULL_ASSERT -std=gnu++2a -o CMakeFiles/main.elf.dir/Src/units.cpp.obj -c /home/diggit/dev/test/Src/units.cpp",
    "file": "/home/diggit/dev/test/Src/units.cpp"
  },

clang

clang -v -fsyntax-only -x c++ /dev/null -target arm-none-eabi                                                                                                                                                                      
clang version 11.0.0
Target: arm-none-unknown-eabi
Thread model: posix
InstalledDir: /usr/bin
 (in-process)
 "/usr/bin/clang-11" -cc1 -triple armv4t-none-unknown-eabi -fsyntax-only -disable-free -disable-llvm-verifier -discard-value-names -main-file-name null -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -fno-verbose-asm -mconstructor-aliases -nostdsysteminc -target-cpu arm7tdmi -target-feature +soft-float-abi -target-feature +strict-align -target-abi aapcs -mfloat-abi soft -fallow-half-arguments-and-returns -fno-split-dwarf-inlining -debugger-tuning=gdb -v -resource-dir /usr/lib/clang/11.0.0 -internal-isystem /usr/lib/clang/11.0.0/include -internal-isystem include -fdeprecated-macro -fdebug-compilation-dir /home/diggit -ferror-limit 19 -fno-signed-char -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -fcolor-diagnostics -faddrsig -x c++ /dev/null
clang -cc1 version 11.0.0 based upon LLVM 11.0.0 default target x86_64-pc-linux-gnu
ignoring nonexistent directory "include"
ignoring duplicate directory "/usr/lib/clang/11.0.0/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/clang/11.0.0/include
End of search list.

gcc

# arm-none-eabi-g++ -v -fsyntax-only -x c++ /dev/null                                                                                                                                                                                
Using built-in specs.
COLLECT_GCC=arm-none-eabi-g++
COLLECT_LTO_WRAPPER=/usr/lib/gcc/arm-none-eabi/10.2.0/lto-wrapper
Target: arm-none-eabi
Configured with: /build/arm-none-eabi-gcc/src/gcc-10.2.0/configure --target=arm-none-eabi --prefix=/usr --with-sysroot=/usr/arm-none-eabi --with-native-system-header-dir=/include --libexecdir=/usr/lib --enable-languages=c,c++ --enable-plugins --disable-decimal-float --disable-libffi --disable-libgomp --disable-libmudflap --disable-libquadmath --disable-libssp --disable-libstdcxx-pch --disable-nls --disable-shared --disable-threads --disable-tls --with-gnu-as --with-gnu-ld --with-system-zlib --with-newlib --with-headers=/usr/arm-none-eabi/include --with-python-dir=share/gcc-arm-none-eabi --with-gmp --with-mpfr --with-mpc --with-isl --with-libelf --enable-gnu-indirect-function --with-host-libstdcxx='-static-libgcc -Wl,-Bstatic,-lstdc++,-Bdynamic -lm' --with-pkgversion='Arch Repository' --with-bugurl=https://bugs.archlinux.org/ --with-multilib-list=rmprofile
Thread model: single
Supported LTO compression algorithms: zlib zstd
gcc version 10.2.0 (Arch Repository) 
COLLECT_GCC_OPTIONS='-v' '-fsyntax-only' '-mcpu=arm7tdmi' '-mfloat-abi=soft' '-marm' '-march=armv4t'
 /usr/lib/gcc/arm-none-eabi/10.2.0/cc1plus -quiet -v -D__USES_INITFINI__ /dev/null -quiet -dumpbase null -mcpu=arm7tdmi -mfloat-abi=soft -marm -march=armv4t -auxbase null -version -fsyntax-only -o /dev/null
GNU C++14 (Arch Repository) version 10.2.0 (arm-none-eabi)
    compiled by GNU C version 10.1.0, GMP version 6.2.0, MPFR version 4.0.2, MPC version 1.1.0, isl version isl-0.22-GMP

warning: MPFR header version 4.0.2 differs from library version 4.1.0.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
ignoring nonexistent directory "/usr/arm-none-eabi/usr/local/include"
ignoring duplicate directory "/usr/arm-none-eabi/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include/c++/10.2.0
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include/c++/10.2.0/arm-none-eabi
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include/c++/10.2.0/backward
 /usr/lib/gcc/arm-none-eabi/10.2.0/include
 /usr/lib/gcc/arm-none-eabi/10.2.0/include-fixed
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include
End of search list.
GNU C++14 (Arch Repository) version 10.2.0 (arm-none-eabi)
    compiled by GNU C version 10.1.0, GMP version 6.2.0, MPFR version 4.0.2, MPC version 1.1.0, isl version isl-0.22-GMP

warning: MPFR header version 4.0.2 differs from library version 4.1.0.
GGC heuristics: --param ggc-min-expand=100 --param ggc-min-heapsize=131072
Compiler executable checksum: 38c7a6c7cc6125453a429785a0230cd0
COMPILER_PATH=/usr/lib/gcc/arm-none-eabi/10.2.0/:/usr/lib/gcc/arm-none-eabi/10.2.0/:/usr/lib/gcc/arm-none-eabi/:/usr/lib/gcc/arm-none-eabi/10.2.0/:/usr/lib/gcc/arm-none-eabi/:/usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/bin/
LIBRARY_PATH=/usr/lib/gcc/arm-none-eabi/10.2.0/:/usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/lib/:/usr/arm-none-eabi/lib/
COLLECT_GCC_OPTIONS='-v' '-fsyntax-only' '-mcpu=arm7tdmi' '-mfloat-abi=soft' '-marm' '-march=armv4t'

It seems, that gcc has some extra paths:

#include <...> search starts here:
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include/c++/10.2.0
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include/c++/10.2.0/arm-none-eabi
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include/c++/10.2.0/backward
 /usr/lib/gcc/arm-none-eabi/10.2.0/include
 /usr/lib/gcc/arm-none-eabi/10.2.0/include-fixed
 /usr/lib/gcc/arm-none-eabi/10.2.0/../../../../arm-none-eabi/include

Is there some automatic way how clangd could query these from gcc? I can copy them manually, but that is not reliable solution....

[diggit]

My bad, just adding this line into config was enough.

"clangd.arguments": ["--query-driver", "/usr/bin/arm-none-eabi-g++"]

[kadircet]

glad to hear that it works, closing the issue then. please re-open if there are any other problems.

[bennowotny]

I have an extremely similar issue:

Problem

I can't get clangd to find the QNX include headers.

Attempted Solution

Adding the qcc path to the query-driver argument did not fix the issue.

Logs

Clangd verbose logs:

...
E[14:34:10.040] System include extraction: start marker not found: cc: looking for gcc_ntox86 in /home/qnx/qnx700/host/linux/x86_64/etc/qcc/gcc/5.4.0/gcc_ntox86++.conf
cc: looking for gcc_ntox86 in /home/qnx/qnx700/host/linux/x86_64/etc/qcc/gcc/5.4.0/gcc_ntox86.conf
/home/qnx/qnx700/host/linux/x86_64/usr/lib/gcc/i586-pc-nto-qnx7.0.0/5.4.0/cc1plus -E -quiet -nostdinc -nostdinc++ -std=gnu++11 -D__QNX__ -D__QNXNTO__ -D__GNUC__=5 -D__GNUC_MINOR__=4 -D__GNUC_PATCHLEVEL__=0 -D__NO_INLINE__ -D__DEPRECATED -D__unix__ -D__unix -D__ELF__ -fPIE -D__X86__ -D__i386__ -mtune=generic -march=i586 -D__LITTLEENDIAN__ -Asystem\(unix\) -isystem /home/qnx/qnx700/target/qnx7/usr/include -isystem /home/qnx/qnx700/host/linux/x86_64/usr/lib/gcc/i586-pc-nto-qnx7.0.0/5.4.0/include -isystem /home/qnx/qnx700/target/qnx7/usr/include/c++/v1 - -o -
cc: removing  /tmp/fileODXNd9_qcc_103745

I[14:34:10.040] ASTWorker building file /home/project/simpleMultiTargetProject/Src/test/test.cpp version 28 with command
[/home/project/simpleMultiTargetProject/Builds/Qnx/Debug/Src/test]
/home/qnx/qnx700/host/linux/x86_64/usr/bin/qcc -Vgcc_ntoaarch64le_gpp -Wc,-isysroot,/home/qnx/qnx700/target/qnx7 -lang-c++ -D_GLIBCXX_USE_C99 -g -O0 -Wno-builtin-macro-redefined -D__FILE__="$(notdir $<)" -std=gnu++14 -o CMakeFiles/test.dir/test.cpp.o -c /home/project/simpleMultiTargetProject/Src/test/test.cpp -fsyntax-only -resource-dir=/home/.config/Code/User/globalStorage/llvm-vs-code-extensions.vscode-clangd/install/12.0.0/clangd_12.0.0/lib/clang/12.0.0
V[14:34:10.045] Ignored diagnostic. -lang-c++: 'linker' input unused
V[14:34:10.046] Driver produced command: cc1 -cc1 -triple x86_64-unknown-linux-gnu -fsyntax-only -disable-free -disable-llvm-verifier -discard-value-names -main-file-name test.cpp -mrelocation-model static -mframe-pointer=all -fmath-errno -fno-rounding-math -mconstructor-aliases -munwind-tables -target-cpu x86-64 -tune-cpu generic -fno-split-dwarf-inlining -debug-info-kind=limited -dwarf-version=4 -debugger-tuning=gdb -resource-dir /home/.config/Code/User/globalStorage/llvm-vs-code-extensions.vscode-clangd/install/12.0.0/clangd_12.0.0/lib/clang/12.0.0 -D _GLIBCXX_USE_C99 -D __FILE__="$(notdir $<)" -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/x86_64-linux-gnu/c++/7.3.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/x86_64-linux-gnu/c++/7.3.0 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/7.3.0/../../../../include/c++/7.3.0/backward -internal-isystem /usr/local/include -internal-isystem /home/.config/Code/User/globalStorage/llvm-vs-code-extensions.vscode-clangd/install/12.0.0/clangd_12.0.0/lib/clang/12.0.0/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -O0 -Wc,-isysroot,/home/qnx/qnx700/target/qnx7 -Wno-builtin-macro-redefined -std=gnu++14 -fdeprecated-macro -fdebug-compilation-dir /home/project/simpleMultiTargetProject/Builds/Qnx/Debug/Src/test -ferror-limit 19 -fgnuc-version=4.2.1 -fcxx-exceptions -fexceptions -faddrsig -x c++ /home/project/simpleMultiTargetProject/Src/test/test.cpp
I[14:34:10.046] --> textDocument/clangd.fileStatus
...
V[14:34:10.049] Ignored diagnostic. unknown warning option '-Wc,-isysroot,/home/qnx/qnx700/target/qnx7'; did you mean '-Wmisleading-indentation'?
...

Something happened between the start of the log and the end of the log that causes clangd to misinterpret how the -Wc command argument should be interpreted, because QNX handles it correctly.

settings.json:

...
    "clangd.arguments": [
        "--query-driver=/home/qnx/qnx700/host/linux/x86_64/usr/bin/q*",
        "--log=verbose",
        "--pretty"
    ]
...

Sample compile_commands.json entry (matches attempted compiling in the clangd logs):

...
    {
    "directory": "/home/project/simpleMultiTargetProject/Builds/Qnx/Debug/Src/test",
    "command": "/home/qnx/qnx700/host/linux/x86_64/usr/bin/qcc -Vgcc_ntoaarch64le_gpp -Wc,-isysroot,/home/qnx/qnx700/target/qnx7 -lang-c++   -D_GLIBCXX_USE_C99 -g -O0 -Wno-builtin-macro-redefined -D__FILE__='\"$(notdir $<)\"'   -std=gnu++14 -o CMakeFiles/test.dir/test.cpp.o -c /home/project/simpleMultiTargetProject/Src/test/test.cpp",
    "file": "/home/project/simpleMultiTargetProject/Src/test/test.cpp"
    }
...

Notes

Surprisingly enough, despite this, an arm-none-eabi-g++ similar toolchain worked correctly and loaded its headers. The problem seems to be limited to qcc.

The project builds, so qcc is working. Clangd appears to have the issue.

Any help?

Clangd Version: 12.0.0

[bennowotny]

@kadircet I think in this case it would be beneficial to re-open the issue.

[kadircet]

Hi @bennowotny

Query-driver is only meant to work with gcc-compatible drivers. If qcc isn't compatible that's unfortunately a known limitation and there's little we can do as specializing for every different driver comes at a cost with diminishing returns.

Can you try invoking /home/qnx/qnx700/host/linux/x86_64/usr/bin/qcc -v -E -xc++ /dev/null and paste the output here? It should produce some sections similar to if qcc is compatible:

#include <...> search starts here:
 /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10
 /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/x86_64-linux-gnu/c++/10
 /usr/lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/backward
 /usr/local/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.

But FWIW, looks like it isn't as -Wc,-isysroot,/home/qnx/qnx700/target/qnx7 isn't a thing. As a workaround you can try post-processing your compile_commands.json to have -isysroot /home/qnx/qnx700/target/qnx7 instead of the -Wc version, which should enable clangd to pick stdlib headers without any need for query-driver (hopefully).

[bennowotny]

It seems that qcc does not have the same output format. Instead, it looks like this:

$/home/qnx/qnx700/host/linux/x86_64/usr/bin/qcc -v -Vgcc_ntoaarch64le_gpp -E -xc++ /dev/null
cc: looking for gcc_ntoaarch64le_gpp in /home/qnx/qnx700/host/linux/x86_64/etc/qcc/gcc/5.4.0/gcc_ntoaarch64le_gpp++.conf
cc: looking for gcc_ntoaarch64le_gpp in /home/qnx/qnx700/host/linux/x86_64/etc/qcc/gcc/5.4.0/gcc_ntoaarch64le_gpp.conf
/home/qnx/qnx700/host/linux/x86_64/usr/lib/gcc/aarch64-unknown-nto-qnx7.0.0/5.4.0/cc1plus -E -quiet -nostdinc -nostdinc++ -std=gnu++11 -D__QNX__ -D__QNXNTO__ -D__GNUC__=5 -D__GNUC_MINOR__=4 -D__GNUC_PATCHLEVEL__=0 -D__NO_INLINE__ -D__DEPRECATED -D__unix__ -D__unix -D__ELF__ -fPIE -D__LITTLEENDIAN__ -Asystem(unix) -idirafter /home/qnx/qnx700/target/qnx7/usr/include -isystem /home/qnx/qnx700/host/linux/x86_64/usr/lib/gcc/aarch64-unknown-nto-qnx7.0.0/5.4.0/include -isystem /home/qnx/qnx700/target/qnx7/usr/include/c++/5.4.0 -isystem /home/qnx/qnx700/target/qnx7/usr/include/c++/5.4.0/aarch64-unknown-nto-qnx7.0.0 -isystem /home/qnx/qnx700/target/qnx7/usr/include/c++/5.4.0/backward -isystem /home/qnx/qnx700/target/qnx7/usr/include -isystem /home/qnx/qnx700/host/linux/x86_64/usr/lib/gcc/aarch64-unknown-nto-qnx7.0.0/5.4.0/include -isystem /home/qnx/qnx700/target/qnx7/usr/aarch64-unknown-nto-qnx7.0.0/include /dev/null -o -
# 1 "/dev/null"
# 1 "<built-in>"
# 1 "<command-line>"
# 1 "/dev/null"

Unfortunately, replacing the -Wc command did not work to fix the include errors. Additionally, I think the -Wc argument is a qcc thing as well. (http://www.qnx.com/developers/docs/6.5.0/index.jsp?topic=%2Fcom.qnx.doc.neutrino_utilities%2Fq%2Fqcc.html)

All of the system includes are there, but they aren't delimited the same way that gcc does it. Additionally, the system includes change based on what system is targeted (For clarity, the command used to generate that output has an extra -Vgcc_ntoaarch64le_gpp argument to select the right target). Is there any way to pipe the cc1plus command or its arguments through to produce the correct output?

[bennowotny]

A new discovery to this: -vv prints the output in the correct format. Is there a way to specify that flag to be run with that driver?

[kadircet]

you can try writing your own wrapper let's say my_wrapper.sh that'll invoke qcc and print the output in gcc-friendly format. afterwards you can change all the mentions of qcc in your compile_commands.json with this wrapper.

[fmkong]

I've made a py script to generate compile_commands.json from QNX build log, please refer to: []https://blog.csdn.net/kongming2liya/article/details/128968589?spm=1001.2014.3001.5501()

[linguini1]

Just wanted to leave this here for other people trying to get correct LSP results using qcc:

I solved the problem by adding -D__QNXNTO__ and -D__LITTLEENDIAN__ to my compile_commands.json file. I also changed all mentions of qcc to aarch64-unknown-nto-qnx7.1.0-gcc, and updated --query-driver to point to .../usr/bin/*gcc instead of .../usr/bin/qcc.

Here's the forum in case you're interested.

[Michaelzhouisnotwhite]

I want to add some information. In vscode just add .clangd file in your project root. Write following to your .clangd file:

# .... other configuration

# Add following
CompileFlags:
  Remove: -Vgcc_ntoaarch64le
  Add:
    - -D__QNXNTO__
    - -D__LITTLEENDIAN__

  Compiler: /<path-to-qnx>/host/linux/x86_64/usr/bin/aarch64-unknown-nto-qnx7.1.0-gcc

# .... other configuration

In this case. You don't have to add --query-driver command argumands.

CAUTION: I've found that sometimes it doesn't work.