"Cannot find compiler" error in leptonica Windows build

[pkgw]

Describe the bug In the conda-forge project, we are trying to build leptonica with sw. The Windows build is failing at the end of the sw setup stage with an error about not being able to find the compiler. (Specifics below.)

Expected behavior No build errors.

To Reproduce Steps to reproduce the behavior:

1. See the build pull request, which has all of the files for the conda-forge CI setup, and associated Azure Pipelines build logs.

Information: Here's the specific message (after some manual linewrapping):

[150/150] C:/Users/VssAdministrator/.sw/storage/tmp/cfg/793929/loc.sw.self.e8bae5-0.0.31.dll
Exception in file D:/dev/cppan2/client2/src/sw/driver/module.cpp:120, function operator ():
  error in module (C:/Users/VssAdministrator/.sw/storage/tmp/cfg/793929/loc.sw.self.4a156b-0.0.31.dll):
    Exception in file D:/dev/cppan2/client2/src/sw/driver/target/native.cpp:799, function findCompiler:
      Cannot find compiler com.Microsoft.VisualStudio.VC.cl->=19.35.0 <19.36.0 for settings: {
        "native": {
          "configuration": "debug", 
          "library": "shared", 
          "mt": "false",
          "program": {
            "asm":"com.Microsoft.VisualStudio.VC.ml->=19.35.0 <19.36.0",
            "c":"com.Microsoft.VisualStudio.VC.cl->=19.35.0 <19.36.0",
            "cpp":"com.Microsoft.VisualStudio.VC.cl->=19.35.0 <19.36.0",
            "lib":"com.Microsoft.VisualStudio.VC.lib->=19.35.0 <19.36.0",
            "link":"com.Microsoft.VisualStudio.VC.link->=19.35.0 <19.36.0"},
          "stdlib": { 
             "c":"com.Microsoft.Windows.SDK.ucrt-10.0.22621.0",
             "cpp":"com.Microsoft.VisualStudio.VC.libcpp->=19.35.0 <19.36.0",
             "kernel":"com.Microsoft.Windows.SDK.um-10.0.22621.0"
           }
        },
       "os": {"arch": "x86_64", "kernel": "com.Microsoft.Windows.NT"}
     }
sw command:
  D:/bld/leptonica_1689029912039/work/build/sw.exe
    -shared
    -platform x64
    -compiler msvc-19.35
    -d D:/bld/leptonica_1689029912039/work/build/.sw/cmake
    -config d,msr,rwdi,r
    integrate
    -cmake-deps D:/bld/leptonica_1689029912039/work/build/.sw/cmake/sw.txt
    -cmake-file-version 7

In our last successful build, instead of the error, the next output was:

Performing 2 check(s): org.sw.demo.jpeg-9.5.0 (jpeg), config 591243
Performing 14 check(s): org.sw.demo.tiff-4.4.0 (tiff), config 591243
Performing 4 check(s): org.sw.demo.uclouvain.openjpeg.openjp2-2.5.0 (openjp2), config 591243

[egorpugin]

Hi,

Do you have cl.exe version 19.35 actually installed? Try add --trace option to sw. You can check what compilers are detected by sw.

[pkgw]

The --trace option doesn't seem to add a whole lot of output, but the information at the top of the build (log here) shows the right version:

2023-07-29T20:21:22.0812748Z Host configuration: {"os":{"arch":"x86_64","kernel":"com.Microsoft.Windows.NT"}}
2023-07-29T20:21:22.0838314Z Registering driver: org.sw.sw.driver.cpp-0.4.1
2023-07-29T20:21:22.2509678Z -- Building for: Visual Studio 17 2022
2023-07-29T20:21:30.0954739Z -- The C compiler identification is MSVC 19.35.32217.1
2023-07-29T20:21:31.5360799Z -- The CXX compiler identification is MSVC 19.35.32217.1

[pkgw]

OK, but when I just execute cl in the build environment, I get:

Microsoft (R) C/C++ Optimizing Compiler Version 19.36.32537 for x64

That's version 19.36, not 19.35. I'll try to figure out why the numbers are disagreeing.

[pkgw]

OK, I believe that our build failure stemmed from those disagreeing version numbers. Due to the way that the conda-forge build environment is set up, the cl found in %PATH% was version 19.36.x, while the version used by MSBuild as invoked by CMake was 19.35.x — it turns out that on Windows in most situations, CMake chooses the compilers based on how MSBuild is set up, and it's very difficult to override. (See below.) I think that the disagreement eventually led to the issue reported above: CMake told sw to look for -compiler msvc-19.35, but then (I am inferring) sw found that cl was a different version.

I don't know enough about the sw framework to gauge whether this should be considered a bug or not. It seems to me that if sw is being launched from CMake, it might be desirable to add some steps to ensure that sw is using the same compiler as CMake/MSBuild, which doesn't appear to be happening at the moment. (Maybe similar problems would happen on other platforms if you set CMAKE_C_COMPILER, etc.?)

Maybe also a message could be added so that if/when this error happens in the future, the user gets a suggestion to check that CMake and sw are using the same compilers.

Also, for posterity: we worked around this problem by setting CMake to use the NMake Makefiles generator and setting CMAKE_C_COMPILER and CMAKE_CXX_COMPILER to the 19.36.x version of cl. My understanding is that the NMake generator is the only one that gives you a way to manually control the compiler selection on Windows — otherwise MSBuild's choice always overrides any attempts to configure the compilers. See the conda-forge PR discussion for a bit more context.

[egorpugin]

Is it possible to run that sw command from cmake from your initial post with -compiler msvc-19.36 (replaced 5 with 6)?

[pkgw]

It would be a challenge for me to reproduce the build environment at this point, but I believe that if I manually invoked it that way, everything would work.