Clangd headers search path on Linux

[alvinhochun]

The clangd binary that comes with llvm-mingw seems to not know where to look for include files. I tried with a Godot build tree that I cross-compiled using llvm-mingw, with the compile_commands.json generated by the build system available, using the Clangd extension on vscode with the path to clangd set to the one inside llvm-mingw, it isn't able to find windows.h.

Running clangd check manually:

/home/alvin/godot-dev/src-mingw$ /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/bin/clangd --check=platform/windows/display_server_windows.cpp
clangd version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
[...]
Loading compilation database...
Loaded compilation database from /home/alvin/godot-dev/src-mingw/compile_commands.json
Compile command from CDB is: [/home/alvin/godot-dev/src-mingw] /usr/bin/x86_64-w64-mingw32-clang++ --target=x86_64-w64-mingw32 --driver-mode=g++ -o platform/windows/display_server_windows.windows.editor.dev.x86_64.llvm.o -c -std=gnu++17 -fno-exceptions -Wa,-mbig-obj -ffp-contract=off -gdwarf-4 -g3 -O0 -Wall -Wshadow-field-in-constructor -Wshadow-uncaptured-local -Wno-ordered-compare-function-pointers -isystem thirdparty/glad -DTOOLS_ENABLED -DDEBUG_ENABLED -DDEV_ENABLED -DENGINE_UPDATE_CHECK_ENABLED -DNO_EDITOR_SPLASH -DWINDOWS_ENABLED -DWASAPI_ENABLED -DWINMIDI_ENABLED -DWINVER=0x0601 -D_WIN32_WINNT=0x0601 -DVULKAN_ENABLED -DRD_ENABLED -DGLES3_ENABLED -DMINGW_ENABLED -DMINGW_HAS_SECURE_API=1 -DMINIZIP_ENABLED -DBROTLI_ENABLED -DTHREADS_ENABLED -DCLIPPER2_ENABLED -DZSTD_STATIC_LINKING_ONLY -DUSE_VOLK -DVK_USE_PLATFORM_WIN32_KHR -DGLAD_ENABLED -DEGL_ENABLED -Ithirdparty/freetype/include -Ithirdparty/libpng -Ithirdparty/volk -Ithirdparty/vulkan -Ithirdparty/vulkan/include -Ithirdparty/zstd -Ithirdparty/zlib -Ithirdparty/clipper2/include -Ithirdparty/brotli/include -Ithirdparty/angle/include -Iplatform/windows -I. -resource-dir=/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18 -- /home/alvin/godot-dev/src-mingw/platform/windows/display_server_windows.cpp
Giving up on broadcasting CDB, as we're shutting down
Parsing command...
internal (cc1) args are: -cc1 -triple x86_64-w64-windows-gnu -fsyntax-only -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name display_server_windows.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=none -fmath-errno -ffp-contract=off -fno-rounding-math -mconstructor-aliases -mms-bitfields -funwind-tables=2 -fno-use-init-array -target-cpu x86-64 -tune-cpu generic -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb -fdebug-compilation-dir=/home/alvin/godot-dev/src-mingw -fcoverage-compilation-dir=/home/alvin/godot-dev/src-mingw -resource-dir /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18 -isystem thirdparty/glad -D TOOLS_ENABLED -D DEBUG_ENABLED -D DEV_ENABLED -D ENGINE_UPDATE_CHECK_ENABLED -D NO_EDITOR_SPLASH -D WINDOWS_ENABLED -D WASAPI_ENABLED -D WINMIDI_ENABLED -D WINVER=0x0601 -D _WIN32_WINNT=0x0601 -D VULKAN_ENABLED -D RD_ENABLED -D GLES3_ENABLED -D MINGW_ENABLED -D MINGW_HAS_SECURE_API=1 -D MINIZIP_ENABLED -D BROTLI_ENABLED -D THREADS_ENABLED -D CLIPPER2_ENABLED -D ZSTD_STATIC_LINKING_ONLY -D USE_VOLK -D VK_USE_PLATFORM_WIN32_KHR -D GLAD_ENABLED -D EGL_ENABLED -I thirdparty/freetype/include -I thirdparty/libpng -I thirdparty/volk -I thirdparty/vulkan -I thirdparty/vulkan/include -I thirdparty/zstd -I thirdparty/zlib -I thirdparty/clipper2/include -I thirdparty/brotli/include -I thirdparty/angle/include -I platform/windows -I . -internal-isystem /usr/x86_64-w64-mingw32/include/c++ -internal-isystem /usr/x86_64-w64-mingw32/include/c++/x86_64-w64-mingw32 -internal-isystem /usr/x86_64-w64-mingw32/include/c++/backward -internal-isystem /usr/x86_64-w64-mingw32/include/c++/ -internal-isystem /usr/x86_64-w64-mingw32/include/c++//x86_64-w64-mingw32 -internal-isystem /usr/x86_64-w64-mingw32/include/c++//backward -internal-isystem /usr/include/c++/ -internal-isystem /usr/include/c++//x86_64-w64-mingw32 -internal-isystem /usr/include/c++//backward -internal-isystem include/c++ -internal-isystem include/c++/x86_64-w64-mingw32 -internal-isystem include/c++/backward -internal-isystem include/g++-v0.0.0 -internal-isystem include/g++-v0.0.0/x86_64-w64-mingw32 -internal-isystem include/g++-v0.0.0/backward -internal-isystem include/g++-v0.0 -internal-isystem include/g++-v0.0/x86_64-w64-mingw32 -internal-isystem include/g++-v0.0/backward -internal-isystem include/g++-v0 -internal-isystem include/g++-v0/x86_64-w64-mingw32 -internal-isystem include/g++-v0/backward -internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18/include -internal-isystem /usr/x86_64-w64-mingw32/include -internal-isystem /usr/x86_64-w64-mingw32/usr/include -O0 -Wall -Wshadow-field-in-constructor -Wshadow-uncaptured-local -Wno-ordered-compare-function-pointers -std=gnu++17 -fdeprecated-macro -ferror-limit 19 -fno-use-cxa-atexit -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -exception-model=seh -no-round-trip-args -faddrsig -x c++ /home/alvin/godot-dev/src-mingw/platform/windows/display_server_windows.cpp
[...]
[pp_file_not_found] Line 31: in included file: 'windows.h' file not found
[...]

First of all, It seems clangd thinks the compiler executable is /usr/bin/x86_64-w64-mingw32-clang++, which does not exist. In the compile_commands.json it is only listed as x86_64-w64-mingw32-clang (I only add llvm-mingw to path while building). This may not really be an issue, but I don't really know. Regardless, I was expecting clangd to be smart enough to see that these executables are right next to itself, but apparently not.

Then, if we look at the list of system includes:

-internal-isystem /usr/x86_64-w64-mingw32/include/c++
-internal-isystem /usr/x86_64-w64-mingw32/include/c++/x86_64-w64-mingw32
-internal-isystem /usr/x86_64-w64-mingw32/include/c++/backward
-internal-isystem /usr/x86_64-w64-mingw32/include/c++/
-internal-isystem /usr/x86_64-w64-mingw32/include/c++//x86_64-w64-mingw32
-internal-isystem /usr/x86_64-w64-mingw32/include/c++//backward
-internal-isystem /usr/include/c++/
-internal-isystem /usr/include/c++//x86_64-w64-mingw32
-internal-isystem /usr/include/c++//backward
-internal-isystem include/c++
-internal-isystem include/c++/x86_64-w64-mingw32
-internal-isystem include/c++/backward
-internal-isystem include/g++-v0.0.0
-internal-isystem include/g++-v0.0.0/x86_64-w64-mingw32
-internal-isystem include/g++-v0.0.0/backward
-internal-isystem include/g++-v0.0
-internal-isystem include/g++-v0.0/x86_64-w64-mingw32
-internal-isystem include/g++-v0.0/backward
-internal-isystem include/g++-v0
-internal-isystem include/g++-v0/x86_64-w64-mingw32
-internal-isystem include/g++-v0/backward
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18/include
-internal-isystem /usr/x86_64-w64-mingw32/include
-internal-isystem /usr/x86_64-w64-mingw32/usr/include

So it is looking for headers under /usr first. I was expecting it to be smart enough to use its built-in headers first, but I guess not.

More surprising is the list of "relative paths" that seem broken. I wonder why they would be included...

Running the build command manually shows the correct system includes:

/home/alvin/godot-dev/src-mingw$ /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/bin/x86_64-w64-mingw32-clang++ -### --target=x86_64-w64-mingw32 --driver-mode=g++ -o platform/windows/display_server_windows.windows.editor.dev.x86_64.llvm.o -c -std=gnu++17 -fno-exceptions -Wa,-mbig-obj -ffp-contract=off -gdwarf-4 -g3 -O0 -Wall -Wshadow-field-in-constructor -Wshadow-uncaptured-local -Wno-ordered-compare-function-pointers -isystem thirdparty/glad -DTOOLS_ENABLED -DDEBUG_ENABLED -DDEV_ENABLED -DENGINE_UPDATE_CHECK_ENABLED -DNO_EDITOR_SPLASH -DWINDOWS_ENABLED -DWASAPI_ENABLED -DWINMIDI_ENABLED -DWINVER=0x0601 -D_WIN32_WINNT=0x0601 -DVULKAN_ENABLED -DRD_ENABLED -DGLES3_ENABLED -DMINGW_ENABLED -DMINGW_HAS_SECURE_API=1 -DMINIZIP_ENABLED -DBROTLI_ENABLED -DTHREADS_ENABLED -DCLIPPER2_ENABLED -DZSTD_STATIC_LINKING_ONLY -DUSE_VOLK -DVK_USE_PLATFORM_WIN32_KHR -DGLAD_ENABLED -DEGL_ENABLED -Ithirdparty/freetype/include -Ithirdparty/libpng -Ithirdparty/volk -Ithirdparty/vulkan -Ithirdparty/vulkan/include -Ithirdparty/zstd -Ithirdparty/zlib -Ithirdparty/clipper2/include -Ithirdparty/brotli/include -Ithirdparty/angle/include -Iplatform/windows -I. -resource-dir=/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18 -- /home/alvin/godot-dev/src-mingw/platform/windows/display_server_windows.cpp
clang version 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
Target: x86_64-w64-windows-gnu
Thread model: posix
InstalledDir: /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/bin
 (in-process)
 "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/bin/clang-18" "-cc1" "-triple" "x86_64-w64-windows-gnu" "-emit-obj" "-mrelax-all" "-disable-free" "-clear-ast-before-backend" "-disable-llvm-verifier" "-discard-value-names" "-main-file-name" "display_server_windows.cpp" "-mrelocation-model" "pic" "-pic-level" "2" "-mframe-pointer=none" "-fmath-errno" "-ffp-contract=off" "-fno-rounding-math" "-mconstructor-aliases" "-mms-bitfields" "-funwind-tables=2" "-fno-use-init-array" "-target-cpu" "x86-64" "-tune-cpu" "generic" "-debug-info-kind=constructor" "-dwarf-version=4" "-debugger-tuning=gdb" "-fdebug-compilation-dir=/home/alvin/godot-dev/src-mingw" "-fcoverage-compilation-dir=/home/alvin/godot-dev/src-mingw" "-resource-dir" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18" "-isystem" "thirdparty/glad" "-D" "TOOLS_ENABLED" "-D" "DEBUG_ENABLED" "-D" "DEV_ENABLED" "-D" "ENGINE_UPDATE_CHECK_ENABLED" "-D" "NO_EDITOR_SPLASH" "-D" "WINDOWS_ENABLED" "-D" "WASAPI_ENABLED" "-D" "WINMIDI_ENABLED" "-D" "WINVER=0x0601" "-D" "_WIN32_WINNT=0x0601" "-D" "VULKAN_ENABLED" "-D" "RD_ENABLED" "-D" "GLES3_ENABLED" "-D" "MINGW_ENABLED" "-D" "MINGW_HAS_SECURE_API=1" "-D" "MINIZIP_ENABLED" "-D" "BROTLI_ENABLED" "-D" "THREADS_ENABLED" "-D" "CLIPPER2_ENABLED" "-D" "ZSTD_STATIC_LINKING_ONLY" "-D" "USE_VOLK" "-D" "VK_USE_PLATFORM_WIN32_KHR" "-D" "GLAD_ENABLED" "-D" "EGL_ENABLED" "-I" "thirdparty/freetype/include" "-I" "thirdparty/libpng" "-I" "thirdparty/volk" "-I" "thirdparty/vulkan" "-I" "thirdparty/vulkan/include" "-I" "thirdparty/zstd" "-I" "thirdparty/zlib" "-I" "thirdparty/clipper2/include" "-I" "thirdparty/brotli/include" "-I" "thirdparty/angle/include" "-I" "platform/windows" "-I" "." "-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++/v1" "-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/include/c++/v1" "-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18/include" "-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include" "-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/usr/include" "-O0" "-Wall" "-Wshadow-field-in-constructor" "-Wshadow-uncaptured-local" "-Wno-ordered-compare-function-pointers" "-std=gnu++17" "-fdeprecated-macro" "-ferror-limit" "19" "-fno-use-cxa-atexit" "-fgnuc-version=4.2.1" "-fskip-odr-check-in-gmf" "-exception-model=seh" "-fcolor-diagnostics" "-faddrsig" "-o" "platform/windows/display_server_windows.windows.editor.dev.x86_64.llvm.o" "-x" "c++" "/home/alvin/godot-dev/src-mingw/platform/windows/display_server_windows.cpp"

Listed:

"-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++/v1"
"-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/include/c++/v1"
"-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18/include"
"-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include"
"-internal-isystem" "/opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/usr/include"

So it seems to be a clangd issue.

---

This does not happen on Windows when the compile_commands.json also don't include an absolute path to the compiler:

D:\dev\godot\godot-src-master> D:\dev\toolchain\llvm-mingw\llvm-mingw-20240606-ucrt-x86_64\bin\clangd.exe --check=platform\windows\display_server_windows.cpp
clangd version 18.1.7 (https://github.com/llvm/llvm-project.git 768118d1ad38bf13c545828f67bd6b474d61fc55)
[...]
Compile command from CDB is: [D:\dev\godot\godot-src-master] C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/bin/x86_64-w64-mingw32-clang++.exe --target=x86_64-w64-mingw32 --driver-mode=g++ -o "platform\\windows\\display_server_windows.windows.editor.dev.x86_64.llvm.san.o" -c -std=gnu++17 -fno-exceptions -Wa,-mbig-obj -ffp-contract=off -fansi-escape-codes -fcolor-diagnostics -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr -gdwarf-4 -g3 -Og -Wall -Wshadow-field-in-constructor -Wshadow-uncaptured-local -Wno-ordered-compare-function-pointers -isystem "thirdparty\\glad" -DTOOLS_ENABLED -DDEBUG_ENABLED -DDEV_ENABLED -DENGINE_UPDATE_CHECK_ENABLED -DNO_EDITOR_SPLASH -DSANITIZERS_ENABLED -DWINDOWS_ENABLED -DWASAPI_ENABLED -DWINMIDI_ENABLED -DWINVER=0x0601 -D_WIN32_WINNT=0x0601 -DVULKAN_ENABLED -DRD_ENABLED -DD3D12_ENABLED -DGLES3_ENABLED -DMINGW_ENABLED -DMINGW_HAS_SECURE_API=1 -DMINIZIP_ENABLED -DBROTLI_ENABLED -DTHREADS_ENABLED -DCLIPPER2_ENABLED -DZSTD_STATIC_LINKING_ONLY -DUSE_VOLK -DVK_USE_PLATFORM_WIN32_KHR -DGLAD_ENABLED -DEGL_ENABLED "-Ithirdparty\\freetype\\include" "-Ithirdparty\\libpng" "-Ithirdparty\\directx_headers\\include\\directx" "-Ithirdparty\\volk" "-Ithirdparty\\vulkan" "-Ithirdparty\\vulkan\\include" "-Ithirdparty\\zstd" "-Ithirdparty\\zlib" "-Ithirdparty\\clipper2\\include" "-Ithirdparty\\brotli\\include" "-Ithirdparty\\angle\\include" "-Iplatform\\windows" -I. "-Ithirdparty\\d3d12ma" -resource-dir=C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/lib/clang/18 -- C:/d_stor/dev/godot/godot-src-master/platform/windows/display_server_windows.cpp
[,,,]
internal (cc1) args are: -cc1 -triple x86_64-w64-windows-gnu -fsyntax-only -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name display_server_windows.cpp -mrelocation-model pic -pic-level 2 -mframe-pointer=none -fmath-errno -ffp-contract=off -fno-rounding-math -mconstructor-aliases -mms-bitfields -funwind-tables=2 -fno-use-init-array -target-cpu x86-64 -tune-cpu generic -debug-info-kind=constructor -dwarf-version=4 -debugger-tuning=gdb "-fdebug-compilation-dir=D:\\dev\\godot\\godot-src-master" "-fcoverage-compilation-dir=D:\\dev\\godot\\godot-src-master" -resource-dir C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/lib/clang/18 -isystem "thirdparty\\glad" -D TOOLS_ENABLED -D DEBUG_ENABLED -D DEV_ENABLED -D ENGINE_UPDATE_CHECK_ENABLED -D NO_EDITOR_SPLASH -D SANITIZERS_ENABLED -D WINDOWS_ENABLED -D WASAPI_ENABLED -D WINMIDI_ENABLED -D WINVER=0x0601 -D _WIN32_WINNT=0x0601 -D VULKAN_ENABLED -D RD_ENABLED -D D3D12_ENABLED -D GLES3_ENABLED -D MINGW_ENABLED -D MINGW_HAS_SECURE_API=1 -D MINIZIP_ENABLED -D BROTLI_ENABLED -D THREADS_ENABLED -D CLIPPER2_ENABLED -D ZSTD_STATIC_LINKING_ONLY -D USE_VOLK -D VK_USE_PLATFORM_WIN32_KHR -D GLAD_ENABLED -D EGL_ENABLED -I "thirdparty\\freetype\\include" -I "thirdparty\\libpng" -I "thirdparty\\directx_headers\\include\\directx" -I "thirdparty\\volk" -I "thirdparty\\vulkan" -I "thirdparty\\vulkan\\include" -I "thirdparty\\zstd" -I "thirdparty\\zlib" -I "thirdparty\\clipper2\\include" -I "thirdparty\\brotli\\include" -I "thirdparty\\angle\\include" -I "platform\\windows" -I . -I "thirdparty\\d3d12ma" -internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/x86_64-w64-mingw32/include/c++/v1 -internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/include/c++/v1 -internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/lib/clang/18/include -internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/x86_64-w64-mingw32/include -internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/x86_64-w64-mingw32/usr/include -internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/include -Og -Wall -Wshadow-field-in-constructor -Wshadow-uncaptured-local -Wno-ordered-compare-function-pointers -std=gnu++17 -fdeprecated-macro -ferror-limit 19 -fmessage-length=119 -fsanitize=address,alignment,array-bounds,bool,builtin,enum,float-cast-overflow,function,integer-divide-by-zero,nonnull-attribute,null,object-size,pointer-overflow,return,returns-nonnull-attribute,shift-base,shift-exponent,signed-integer-overflow,unreachable,vla-bound -fsanitize-recover=alignment,array-bounds,bool,builtin,enum,float-cast-overflow,function,integer-divide-by-zero,nonnull-attribute,null,object-size,pointer-overflow,returns-nonnull-attribute,shift-base,shift-exponent,signed-integer-overflow,vla-bound -fsanitize-system-ignorelist=C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/lib/clang/18/share/asan_ignorelist.txt -fno-sanitize-memory-param-retval -fsanitize-address-use-after-scope -fsanitize-address-globals-dead-stripping -fno-sanitize-address-use-odr-indicator -fno-assume-sane-operator-new -fno-use-cxa-atexit -fgnuc-version=4.2.1 -fskip-odr-check-in-gmf -exception-model=seh -fansi-escape-codes -no-round-trip-args -faddrsig -x c++ C:/d_stor/dev/godot/godot-src-master/platform/windows/display_server_windows.cpp
[...]

The list of system includes: (D:\ gets changed to C:\d_stor due to path canonicalization, which is another thing that bothers me but let's ignore that now):

-internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/x86_64-w64-mingw32/include/c++/v1
-internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/include/c++/v1
-internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/lib/clang/18/include
-internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/x86_64-w64-mingw32/include
-internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/x86_64-w64-mingw32/usr/include
-internal-isystem C:/d_stor/dev/toolchain/llvm-mingw/llvm-mingw-20240606-ucrt-x86_64/include

If you want to reproduce this, it may be simpler to craft a compile_commands.json file manually, but the build command I used on Linux is scons -j2 dev_build=yes platform=windows use_mingw=yes use_llvm=yes target=editor.

[alvinhochun]

Even when I patched compile_commands.json manually to include the full path to the toolchain, clangd still seems very confused:

-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++/x86_64-w64-mingw32
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++/backward
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++/
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++//x86_64-w64-mingw32
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++//backward
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/include/c++/
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/include/c++//x86_64-w64-mingw32
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/include/c++//backward
-internal-isystem include/c++
-internal-isystem include/c++/x86_64-w64-mingw32
-internal-isystem include/c++/backward
-internal-isystem include/g++-v0.0.0
-internal-isystem include/g++-v0.0.0/x86_64-w64-mingw32
-internal-isystem include/g++-v0.0.0/backward
-internal-isystem include/g++-v0.0
-internal-isystem include/g++-v0.0/x86_64-w64-mingw32
-internal-isystem include/g++-v0.0/backward
-internal-isystem include/g++-v0
-internal-isystem include/g++-v0/x86_64-w64-mingw32
-internal-isystem include/g++-v0/backward
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18/include
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/usr/include

None of the paths include the c++/v1 subdirectory, so it still cannot find a lot of headers, for example cstdint.

[mstorsjo]

I'm not entirely sure all what's happening here, but I have a few observations at least.

When we invoke <arch>-w64-mingw32-clang, the wrapper script/executable invokes the clang binary with a couple of extra parameters - -target <arch>-w64-mingw32, but also -stdlib=libc++ which may be relevant here.

When clangd sees an invocation of <arch>-w64-mingw32-clang, it seems like it is able to deduce that this in general implies -target <arch>-w64-mingw32. (Instead of a wrapper, one could also just make a symlink to the clang executable, and have it imply the target that way.) But clangd can't know that the wrapper also implies -stdlib=libc++. Adding that to the command in compile_commands.json might help (but that's not a proper fix though).

This explains the difference to the Windows build of llvm-mingw; in the Windows build, we have such parameters hardcoded in the clang binary (see https://github.com/mstorsjo/llvm-mingw/blob/master/build-llvm.sh#L242-L251). However I'm not sure if this aspect explains why it thinks that things are based in /usr.

I'm not very familiar with the clangd code - I assume that it should run the target specific logic for setting up include paths in clang/lib/Driver/ToolChains/MinGW.cpp just like it otherwise does, but perhaps there's some difference in how it decides what the base directory is?

---

See https://github.com/mstorsjo/llvm-mingw/issues/429#issuecomment-2154595788 for a discussion of similar issues with clang-tidy, discussions on a new wrapper for clang-scan-deps in https://github.com/mstorsjo/llvm-mingw/pull/425, and https://github.com/mstorsjo/llvm-mingw/pull/430 which reveals such hidden parameters if using CMake (but goes against the way this toolchain mostly is used).

Another way around this would be to use config files for setting most of these parameters. I made a PoC for that many years ago, perhaps I should revisit it and see how well it would work, compared to our wrappers, at this point.

[alvinhochun]

Your guess is right, adding -stdlib=libc++ gets me this list:

-internal-isystem /usr/x86_64-w64-mingw32/include/c++/v1
-internal-isystem /usr/include/c++/v1
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18/include
-internal-isystem /usr/x86_64-w64-mingw32/include
-internal-isystem /usr/x86_64-w64-mingw32/usr/include

And adding the full path to the compiler executables does produce fully working system includes:

-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include/c++/v1
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/include/c++/v1
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/lib/clang/18/include
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/include
-internal-isystem /opt/llvm-mingw-20240619-ucrt-ubuntu-20.04-x86_64/x86_64-w64-mingw32/usr/include

I'm not very familiar with the clangd code - I assume that it should run the target specific logic for setting up include paths in clang/lib/Driver/ToolChains/MinGW.cpp just like it otherwise does, but perhaps there's some difference in how it decides what the base directory is?

The documentation does say it uses the toolchains in clang/lib/Driver/ToolChains/ to do the search.

It also says there is a --query-driver mode where it can ask the compiler executable for the flags, which I suppose might fix the issue if the executables use full paths. But this requires manual setup so it is an inconvenient workaround.

I think it will be nice if clangd can have logic to automatically detect the compiler path in the same directory, then also have some mingw-specific logic to detect that the toolchain only ships with libc++ and automatically use that. Though I don't know if it can be added to clangd without affecting clang. Or perhaps clang could also benefit from this logic?

[mstorsjo]

Hmm, so to get the full picture - when you generate your compile_commands.json, you have the llvm-mingw tools added to $PATH, and your build system, which generates the json file, just encode the commands with the plain executable names, without an absolute path - expecting to find them in $PATH. And when you run clangd, elsewhere, they don't exist in the $PATH of clangd?

I think this bit mostly is operating as expected actually.

I think it's correct design that clangd doesn't assume that the base installation directory of clangd would be related in any way to the base installation directory of the compilers used. In many cases, clangd would be supplied entirely separately (e.g. installed as part of a VSCode plugin) from the actual toolchains that you may use. So I think it's actually an intentional design, that clangd doesn't try to look for compiler executables in the directory it is installed in. I.e., if the tools aren't generally found in the $PATH of clangd, then the build system should generate compile_commands.json with absolute paths. (E.g. CMake in general always uses absolute paths to the compilers.)

[alvinhochun]

But it appears to work on Windows without the path. Why would there be a discrepancy?

[mstorsjo]

But it appears to work on Windows without the path. Why would there be a discrepancy?

That's indeed odd. Perhaps the last-call default fallbacks differ - on Unix assuming /usr/bin if nothing else is found, but assuming the current executable directory on Windows?

[alvinhochun]

Just to be clear, Clangd relies on heuristics on some level to find stuff (it's stated in its documentation), so it seems to me that, if the compiler driver is not found on PATH (which usually includes /usr/bin), the same directory itself should be a reasonable next best guess. After all, this will only have an effect if the copy of Clangd is included with a standalone toolchain, and it seems unlikely that a user will just randomly decide to use a Clangd supplied with a cross toolchain (if normal cross toolchains even come with Clangd). Or perhaps it can be added as an option when compiling Clangd.

But I guess this ultimately needs to be discussed with Clangd developers.

[mstorsjo]

Just to be clear, Clangd relies on heuristics on some level to find stuff (it's stated in its documentation), so it seems to me that, if the compiler driver is not found on PATH (which usually includes /usr/bin), the same directory itself should be a reasonable next best guess. After all, this will only have an effect if the copy of Clangd is included with a standalone toolchain, and it seems unlikely that a user will just randomly decide to use a Clangd supplied with a cross toolchain (if normal cross toolchains even come with Clangd). Or perhaps it can be added as an option when compiling Clangd.

Yeah I guess that could be reasonable. But it would be interesting to dig into the code and see what it actually does and why the behaviour differs (or why it seems to find it on Windows - did you happen to have that path added when you started clangd there?)

[alvinhochun]

did you happen to have that path added when you started clangd there?

Nope, I never add toolchains on PATH except in the terminals I actively use to compile things.

[mstorsjo]

did you happen to have that path added when you started clangd there?

Nope, I never add toolchains on PATH except in the terminals I actively use to compile things.

Another theory could be if it finds a clang binary in /usr/bin and uses that as reference, but I'm not quite sure why it would look for that, when the actual executable mentioned in the json file is <triple>-clang.

[mstorsjo]

did you happen to have that path added when you started clangd there?

Nope, I never add toolchains on PATH except in the terminals I actively use to compile things.

Another theory could be if it finds a clang binary in /usr/bin and uses that as reference, but I'm not quite sure why it would look for that, when the actual executable mentioned in the json file is <triple>-clang.

Yes, this actually seems to be the case.

See https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0-rc1/clang-tools-extra/clangd/CompileCommands.cpp#L88-L118 and https://github.com/llvm/llvm-project/blob/llvmorg-19.1.0-rc1/clang-tools-extra/clangd/CompileCommands.cpp#L154-L172

[nolange]

As far as I my experience goes - gnu(-like) compilers will assume they should use host system paths, unless you add --sysroot, at which point the paths will be re-rooted there (among other things). clangd will only look at the arguments, and not inspect the compiler AFAIK. If those are hidden then it can only use blind guesses.

This is a general issue with alot clang-tooling using compile_commands.json (ex. Codechecker too).

if you use CMake as buildsystem, using toolchain files (#430) will ensure everything works.