build system somehow finds yosys not on path, and then fails to run it

[ArcaneNibble]

$ cmake --version
cmake version 3.9.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).

I'm not sure what changed, but the build system is once again failing to fail to find yosys. I have a random ancient copy of yosys installed under /opt which is not on my PATH. Somehow, cmake finds it and manages to set ${YOSYS} to the path to it. However, because it isn't on my PATH, it then fails to run obviously and then causes build failures.

[ArcaneNibble]

--verbose did not give any extra information, but this issue only happens if I try to cross compile using cmake --verbose .. -DBUILD_RUST=off -DBUILD_DOC=off -DCMAKE_INSTALL_PREFIX=/opt/fpgatools -DCMAKE_C_COMPILER=x86_64-linux-musl-gcc -DCMAKE_CXX_COMPILER=x86_64-linux-musl-g++ -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_C_FLAGS="--sysroot=/home/rqou/code/fpgatools-build-system/tools-x86_64/sysroot" -DCMAKE_CXX_FLAGS="--sysroot=/home/rqou/code/fpgatools-build-system/tools-x86_64/sysroot" -DCMAKE_EXE_LINKER_FLAGS="-static"

A normal cmake .. does not trigger this problem.

[ArcaneNibble]

cc @whitequark

[whitequark]

@rqou You're not using cmake properly. See cmake-toolchains.

[ArcaneNibble]

I updated my cmake command based on that, and it still falsely detects yosys (command is cmake --verbose .. -DBUILD_RUST=off -DBUILD_DOC=off -DCMAKE_INSTALL_PREFIX=/opt/fpgatools -DCMAKE_C_COMPILER=x86_64-linux-musl-gcc -DCMAKE_CXX_COMPILER=x86_64-linux-musl-g++ -DCMAKE_SYSTEM_NAME=Linux -DCMAKE_SYSROOT="/home/rqou/code/fpgatools-build-system/tools_x86_64/sysroot" -DCMAKE_EXE_LINKER_FLAGS="-static" -DCMAKE_FIND_ROOT_PATH_MODE_PROGRAM=NEVER -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=ONLY -DCMAKE_FIND_ROOT_PATH_MODE_PACKAGE=ONLY

Instead, it seems the problem is CMAKE_INSTALL_PREFIX. If I change it to e.g. /opt/fpgatools2 then yosys is correctly not found as expected. Somehow cmake is deciding to look in there to find tools even though it's not on PATH. Is this just a cmake quirk or is this some other issue?

[whitequark]

@rqou You're not using cmake properly. You need to read documentation more carefully.

If cmake(1) is invoked with the command line parameter -DCMAKE_TOOLCHAIN_FILE=path/to/file,

So, these variables should be put into a file, which is loaded before the rest of CMake internals are initialized. Not on the command line. What is it with your love for shitty ad-hoc solutions?

[ArcaneNibble]

I figured that since I have to provide command-line arguments anyways (e.g. CMAKE_INSTALL_PREFIX), I might as well provide all the settings on the command line. Creating a separate file seems pointless. Anyways, I used a separate file and it didn't fix the issue:

$ cat ../testtoolchain 
set(CMAKE_SYSTEM_NAME Linux)

set(CMAKE_SYSROOT /home/rqou/code/fpgatools-build-system/tools-x86_64/sysroot)

set(CMAKE_C_COMPILER x86_64-linux-musl-gcc)
set(CMAKE_CXX_COMPILER x86_64-linux-musl-g++)

set(CMAKE_EXE_LINKER_FLAGS "-static")

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

$ cmake --verbose .. -DBUILD_RUST=off -DBUILD_DOC=off -DCMAKE_INSTALL_PREFIX=/opt/fpgatools -DCMAKE_TOOLCHAIN_FILE=../testtoolchain 
-- The C compiler identification is GNU 6.3.0
-- The CXX compiler identification is GNU 6.3.0
-- Check for working C compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-gcc
-- Check for working C compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-g++
-- Check for working CXX compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- BUILD_DOC turned off, documentation will not be built
-- BUILD_RUST turned off, Rust code will not be built nor run
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rqou/code/openfpga/build2

$ cmake --verbose .. -DBUILD_RUST=off -DBUILD_DOC=off -DCMAKE_INSTALL_PREFIX=/opt/fpgatools2 -DCMAKE_TOOLCHAIN_FILE=../testtoolchain 
-- The C compiler identification is GNU 6.3.0
-- The CXX compiler identification is GNU 6.3.0
-- Check for working C compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-gcc
-- Check for working C compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-g++
-- Check for working CXX compiler: /home/rqou/code/fpgatools-build-system/tools-x86_64/toolchain/bin/x86_64-linux-musl-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- BUILD_DOC turned off, documentation will not be built
CMake Warning at CMakeLists.txt:88 (message):
  yosys not found, tests will not be built nor run

-- BUILD_RUST turned off, Rust code will not be built nor run
-- Configuring done
-- Generating done
-- Build files have been written to: /home/rqou/code/openfpga/build2

So having files under CMAKE_INSTALL_PREFIX is still causing something unexpected to happen.

[whitequark]

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)

CMAKE_FIND_ROOT_PATH_MODE_PROGRAM This variable controls whether the CMAKE_FIND_ROOT_PATH and CMAKE_SYSROOT are used by find_program().

If set to ONLY, then only the roots in CMAKE_FIND_ROOT_PATH will be searched. If set to NEVER, then the roots in CMAKE_FIND_ROOT_PATH will be ignored and only the host system root will be used. If set to BOTH, then the host system paths and the paths in CMAKE_FIND_ROOT_PATH will be searched.

You've specifically told cmake to only use the host (i.e. CMAKE_INSTALL_PREFIX) programs...

[ArcaneNibble]

Right, but the cmake "host system root" should not include CMAKE_INSTALL_PREFIX. Especially since CMAKE_INSTALL_PREFIX isn't in PATH, CMAKE_INSTALL_PREFIX can contain basically anything.

[whitequark]

See the documentation on CMAKE_FIND_ROOT_PATH (from find_path):

The CMake variable CMAKE_FIND_ROOT_PATH specifies one or more directories to be prepended to all other search directories. This effectively “re-roots” the entire search under given locations.

[ArcaneNibble]

But I've specifically told cmake not to use CMAKE_FIND_ROOT_PATH? And I don't see anything about searching under CMAKE_INSTALL_PREFIX.

[ArcaneNibble]

Er, just to be clear we're talking about the same thing, I'm complaining that cmake is failing to fail to find something, not that cmake is failing to find something.

[ArcaneNibble]

My understanding is that CMAKE_INSTALL_PREFIX should control where build outputs go after they are installed. This might not even be on the same machine, especially when cross-compiling. This is why I'm confused why cmake is searching under CMAKE_INSTALL_PREFIX at build time. On the build machine, this path has no meaning whatsoever (and might not even exist), so searching for host binaries there (given that it's not on PATH) seems totally wrong.

[awygle]

Both CMAKE_SYSTEM_PREFIX_PATH and CMAKE_INSTALL_PREFIX state that CMAKE_INSTALL_PREFIX is added to the search path for find_library etc. Is that what you're being surprised by?

[ArcaneNibble]

Yes, that's it. I missed that because CMAKE_SYSTEM_PREFIX_PATH is not the same as CMAKE_FIND_ROOT_PATH, and I didn't see anything in the docs for CMAKE_FIND_ROOT_PATH pointing to CMAKE_SYSTEM_PREFIX_PATH. Now I just need a way to make this not happen...

[ArcaneNibble]

Alright, adding -DCMAKE_FIND_NO_INSTALL_PREFIX=TRUE makes this issue fixed.