Sporadic failure on Windows with `FindFirstFileExA`

[compnerd]

ninja: error: FindFirstFileExA(note: including file: d:/a/swift-build/swift-build/sourcecache/swift-corelibs-foundation/corefoundation): The filename, directory name, or volume label syntax is incorrect.

I've been unable to isolate the exact scenario that this happens on, but I've seen this happen periodically on GitHub Actions builders recently. It has also happened a couple of times on Azure DevOps builders. The build rules don't seem to have changed, so I suspect that this might be related to a Windows update (the builders do update periodically).

[michael-brade]

Happens to me whenever I tell cmake to use clang-cl instead of msvc on Windows. The problem is that the file name it wants to find is called "note: inluding file: c:/...." and obviously, that is not a proper filename. I replaced all "note: " strings in my repository but still get this error. So I don't know where it comes from.

And btw, if I change the generator to "MinGW Makefiles", it works. So this must be a ninja problem.

[fischman-bcny]

Seems related to #2138 and #1037. @compnerd in #1239 nico suggests that flushing lines emitted to stdout/stderr from the tools driven by ninja might avoid this issue. Assuming y/our problem is in driving clang-cl per @michael-brade's comment above, WDYT of making clang-cl flush on newlines?

[compnerd]

I would rather not do that in clang-cl. clang-cl is already ~10-15% slower than cl, this would further slow down the already slow compiler. Perhaps a benchmark on Windows of chromium w-w/o the change to clang-cl would be helpful to judge the impact.

[tristanlabelle]

I think this is a CMake bug. When I ran into this issue, my rules.ninja file generated by CMake contained:

command = C:/PROGRA~1/MICROS~2/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/CMake/bin/cmcldeps.exe RC $in $DEP_FILE $out "" "S:/b/1/bin/clang-cl.exe" C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\rc.exe $DEFINES $INCLUDES $FLAGS /fo $out $in

where "" (after $out) should have been the detected prefix Note: including file:. The CMakeFiles\3.26.0-msvc3\CMakeCCompiler.cmake file contained:

set(CMAKE_C_CL_SHOWINCLUDES_PREFIX "")
if(CMAKE_C_CL_SHOWINCLUDES_PREFIX)
  set(CMAKE_CL_SHOWINCLUDES_PREFIX "${CMAKE_C_CL_SHOWINCLUDES_PREFIX}")
endif()

Which seems to point to a bug in CMake's CMAKE_DETERMINE_MSVC_SHOWINCLUDES_PREFIX.

However, it would help if ninja supported parsing dependencies from a more reliable format like /sourceDependencies//scanDependencies

[michael-brade]

I can't confirm that. My rules.ninja doesn't have any "" in it, and the cmcldeps call always has the "Note: including.." right after $out. However, I also changed the string to "Note: X including..." and the error message by ninja still stays the same.

[compnerd]

https://github.com/ninja-build/ninja/issues/2159 might also be related to this.

[michael-brade]

I thought so, too, but I compiled ninja HEAD and the error really is the wrong filename, not the directory error.

[KenSykes]

I am consistently hitting this and it is always the RC command:

command = C:/PROGRA~1/MICROS~4/2022/ENTERP~1/Common7/IDE/COMMON~1/MICROS~1/CMake/CMake/bin/cmcldeps.exe RC $in $DEP_FILE $out "" "C:/Program Files/Microsoft Visual Studio/2022/Enterprise/VC/Tools/Llvm/x64/bin/clang-cl.exe" C:\PROGRA~2\WI3CF2~1\10\bin\100226~1.0\x64\rc.exe $DEFINES $INCLUDES $FLAGS /fo $out $in

If I delete the cache and reconfigure the build works once. Build All / Rebuild All after that build goes back to the FindFirstFileExA error.

If it helps I have a Time Travel Debugging trace of ninja.exe producing this error. It's about 11MB as a .zip. You could debug this by opening the trace in WinDbg, doing !tt 4044:1F93, and then step backwards to see how ninja ended up with this path.

0:000> dx -g @$cursession.TTD.Calls(kernelbase!FindFirstFileExA).Select(x => new { Position = x.TimeStart, FileName = x.Parameters.lpFileName, ReturnValue = x.ReturnValue})
===================================================================================================================
=           = (+) Position = (+) FileName                                                 = (+) ReturnValue       =
===================================================================================================================
= [0x0]     - 3B09:4BF     - 0x1000ff080 : ".\*"                                          - 0x16dba901150         =
= [0x1]     - 3B17:15AE    - 0x1000ff000 : "External\*"                                   - 0x16dba901150         =
= [0x2]     - 3B22:17C3    - 0x16dba8eab90 : "External/Msatx\*"                           - 0x16dba901990         =
...
= [0x45]    - 4044:1F93    - 0x16dbae0ca30 : "Note: including file:    C:/Program File... - 0xffffffffffffffff    =
===================================================================================================================

0:000> !tt 4044:1F93 

(c938.43c0): Break instruction exception - code 80000003 (first/second chance not available)
Time Travel Position: 4044:1F93

0:000> kP1
 # Child-SP          RetAddr               Call Site
00 00000001`000fee18 00007ff6`04afbd28     KERNELBASE!FindFirstFileExA(
            char * lpFileName = 0x0000016d`bae0ca30 "Note: including file:    C:/Program Files (x86)/Windows Kits/10/include/10.0.22621.0/shared\*", 
            _FINDEX_INFO_LEVELS fInfoLevelId = FindExInfoBasic (0n1), 
            void * lpFindFileData = 0x00000001`000fefb0, 
            _FINDEX_SEARCH_OPS fSearchOp = FindExSearchNameMatch (0n0), 
            void * lpSearchFilter = 0x00000000`00000000, 
            unsigned long dwAdditionalFlags = 0) [minkernel\kernelbase\filefind.c @ 591] 

[hsdk123]

+1 - can confirm all above

[hsdk123]

For Windows, I wonder if it has to do with case insensitivity in Windows. I notice ninja trying to reference include, when I notice the folder actually named Include. This only seems to happen when using clang-cl with me.

Ex. C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\shared is attempted to be loaded as C:\Program Files (x86)\Windows Kits\10\include\10.0.19041.0\shared (Note the smaller case i in what should be Include)

[tsteven4]

I can reproduce with Ninja 1.11.0

Severity    Code    Description Project File    Line    Suppression State
Error       FindFirstFileExA(Note: including file:    C:/Program Files (x86)/Windows Kits/10/include/10.0.22000.0/shared): The filename, directory name, or volume label syntax is incorrect.   C:\Users\user1\source\repos\whit2\out\build\x64-Clang-Debug\whit2   C:\Users\user1\source\repos\whit2\out\build\x64-Clang-Debug\ninja   1   

[tsteven4]

If i edit CMakeFiles/rules.ninja after "Delete Cache and Reconfigure", but before "Build All" then I can clean and rebuild repeatedly.

$ diff CMakeFiles/rules.ninja ../../..
38c38
<   command = C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/CMake/bin/cmcldeps.exe RC $in $DEP_FILE $out "" "C:/PROGRAM FILES/MICROSOFT VISUAL STUDIO/2022/COMMUNITY/VC/Tools/Llvm/x64/bin/clang-cl.exe" C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\rc.exe $DEFINES $INCLUDES $FLAGS /fo $out $in
---
>   command = C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/CMake/bin/cmcldeps.exe RC $in $DEP_FILE $out "Note: including file:" "C:/PROGRAM FILES/MICROSOFT VISUAL STUDIO/2022/COMMUNITY/VC/Tools/Llvm/x64/bin/clang-cl.exe" C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\rc.exe $DEFINES $INCLUDES $FLAGS /fo $out $in
128c128
<   command = C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/CMake/bin/cmcldeps.exe RC $in $DEP_FILE $out "" "C:/PROGRAM FILES/MICROSOFT VISUAL STUDIO/2022/COMMUNITY/VC/Tools/Llvm/x64/bin/clang-cl.exe" C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\rc.exe $DEFINES $INCLUDES $FLAGS /fo $out $in
---
>   command = C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/CMake/bin/cmcldeps.exe RC $in $DEP_FILE $out "Note: including file:" "C:/PROGRAM FILES/MICROSOFT VISUAL STUDIO/2022/COMMUNITY/VC/Tools/Llvm/x64/bin/clang-cl.exe" C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\rc.exe $DEFINES $INCLUDES $FLAGS /fo $out $in

[tsteven4]

I am using cmake version 3.26.0-msvc3

[KenSykes]

so looks like a CMake issue, caused potentially by change in output of msvc /showIncludes?

[tsteven4]

Note that my unmodified and modified rules.ninja file has a line "deps = gcc", so /showIncludes shouldn't be in play.

#############################################
# Rule for compiling RC files.

rule RC_COMPILER__gpsbabel_unscanned_Debug
  depfile = $DEP_FILE
  deps = gcc
  command = C:/PROGRA~1/MICROS~4/2022/COMMUN~1/Common7/IDE/COMMON~1/MICROS~1/CMake/CMake/bin/cmcldeps.exe RC $in $DEP_FILE $out "" "C:/PROGRAM FILES/MICROSOFT VISUAL STUDIO/2022/COMMUNITY/VC/Tools/Llvm/x64/bin/clang-cl.exe" C:\PROGRA~2\WI3CF2~1\10\bin\100220~1.0\x64\rc.exe $DEFINES $INCLUDES $FLAGS /fo $out $in
  description = Building RC object $out

[tsteven4]

A workaround is to define a cmake variable in the CMake command arguments box: -DCMAKE_CL_SHOWINCLUDES_PREFIX="Note: including file:"

The build system generation will create a ninja.rules with the "" -> "Note: including file:"

[tsteven4]

I believe this was fixed in https://github.com/microsoft/CMake/commit/843fc607de7 which first appeared in cmake v3.26.3-msvc1. But MS is currently distributing cmake v3.26.0-msvc3.

Thus, this is not an ninja issue, but an issue with cmake. Furthermore it has been fixed, but MS hasn't distributed the fixed executable yet.

[tsteven4]

I reported this to MS: https://developercommunity.visualstudio.com/t/cmake-generates-bad-dependencies-for-rc/10398924?q=cmake+dependencies

[michael-brade]

great discovery! If I use stock cmake 3.26.4, I can finally compile with ninja and clang-cl :)

[guijiyang]

I still got this error with cmake 3.28.3

ninja: error: FindFirstFileExA(Note: including file:     C:/Program Files (x86)/Windows Kits/10/include/10.0.22621.0/shared): The filename, directory name, or volume label syntax is incorrect.

[IGR2014]

Well, CMake 3.29.2, Clang 18.1.0, and guess why am I here?)

ninja : error : FindFirstFileExA(Note: including file:      C:/Program Files (x86)/Windows Kits/10/include/10.0.22621.0/shared): The filename, directory name, or volume label syntax is incorrect.

[digit-google]

This file path comes from using /showIncludes when invoking your compiler. This forces the compiler to print it with the Note: include file: prefix in stdout, and using deps = msvc in your build plan will force Ninja to parse stdout to extract the file path after the prefix.

However, this prefix is localized, and just can change between, say, and English or Chinese locale. So there is also msvc_deps_prefix that you can set explicitly to care for this.Yes, it is a horrible hack, and we'd better solve this with a different solution (see Issue #1806)

CMake seems to have been confused and produced an invalid build plan. Or Ninja is very confused by something. If you could provide a simple reproduction case, that would be very helpful.

See https://ninja-build.org/manual.html#_depfile and https://learn.microsoft.com/en-us/previous-versions/visualstudio/visual-studio-2008/hdkef6tk(v=vs.90)?redirectedfrom=MSDN

[IGR2014]

This exact issue happened to me ONLY when I was trying to install what was built. The build stage itself compelted fine

[weypro]

I encountered a similar error:

ninja: error: FindFirstFileExA(Note: including file:      C:/Program Files (x86)/Windows Kits/10/include/10.0.26100.0/shared):...

My clang-cl --version output is as follows:

clang version 18.1.7
Target: x86_64-pc-windows-msvc
Thread model: posix

Upon further investigation, I discovered a Ninja build system error related to the .ninja_deps file. This error occurs when processing dependencies for Windows resource files (.rc files).

In CMake, .rc files can be built as part of the source code, allowing the generated dynamic library to have specific attribute information.

Even when simplifying the .rc file to contain only a single #include "winres.h" statement, the error persists. This may indicate an issue with Ninja parsing or storing paths for certain types of dependencies.

Here's a minimal example:

// version.rc
#include "winres.h"

# CMakeLists.txt
set(MYLIB_SOURCES library.cpp)
list(APPEND MYLIB_SOURCES "${CMAKE_BINARY_DIR}/version.rc")
add_library(dllversionlibtest SHARED ${MYLIB_SOURCES})

// library.cpp
#include "library.h"
#include <iostream>
void hello() {
    std::cout << "Hello, World!" << std::endl;
}

I'm not sure about others' experiences, but for this example, the error doesn't occur on the first build. The first build is successful, but the error appears after the .ninja_deps file is generated.

I have limited knowledge of Ninja and clang-cl, but I seem to have found an issue:

https://github.com/ninja-build/ninja/blob/dcefb838534a56b2621baf33c7e0583ff63ee35f/src/build.cc#L938

The deps here come from CMakeFiles/rules.ninja.

Its result is "gcc". Could this be one of the reasons for the incorrect judgment?When I modified the Ninja code to keep only the Builder::ExtractDeps parsing code for msvc, the recompiled Ninja was able to successfully build the above example.

[wantehchang]

According to https://gitlab.kitware.com/cmake/cmake/-/issues/25753, this bug has been fixed in cmake 3.29.