CMake Error at cmake/modules/CMakeLists.txt

[shoober420]

I'm not sure which commit caused this, but when compiling against LLVM master branch, I get the following errors. Full logs attached.

Build Log:

CMake Error at cmake/modules/CMakeLists.txt:1 (include):
  include could not find requested file:

    ExtendPath

CMake Error at cmake/modules/CMakeLists.txt:3 (include):
  include could not find requested file:

    FindPrefixFromConfig

CMake Error at cmake/modules/CMakeLists.txt:113 (find_prefix_from_config):
  Unknown CMake command "find_prefix_from_config".

CMakeError Log:

Run Build Command(s):/usr/bin/ninja cmTC_ba7f6 && [1/2] Building CXX object CMakeFiles/cmTC_ba7f6.dir/src.cxx.o
FAILED: CMakeFiles/cmTC_ba7f6.dir/src.cxx.o 
/usr/bin/x86_64-pc-linux-gnu-g++ -m32 -DHAS_WERROR_GLOBAL_CTORS   -DNDEBUG -O2 -flarge-source-files -w -fPIC -fno-semantic-interposition -fvisibility-inlines-hidden -Werror=date-time -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess -Wno-redundant-move -Wno-pessimizing-move -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wmisleading-indentation -fdiagnostics-color -ffunction-sections -fdata-sections    -Werror=global-constructors -std=c++14 -o CMakeFiles/cmTC_ba7f6.dir/src.cxx.o -c /var/tmp/portage/sys-devel/llvm-14.0.0.9999/work/llvm-14.0.0.9999_build-abi_x86_32.x86/CMakeFiles/CMakeTmp/src.cxx
cc1plus: error: '-Werror=global-constructors': no option '-Wglobal-constructors'; did you mean '-felide-constructors'?
ninja: build stopped: subcommand failed.

Source file was:
int main() { return 0; }

CMakeOutput Log:

Performing C++ SOURCE FILE Test CXX_SUPPORTS_FDATA_SECTIONS succeeded with the following output:
Change Dir: /var/tmp/portage/sys-devel/llvm-14.0.0.9999/work/llvm-14.0.0.9999_build-abi_x86_32.x86/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/ninja cmTC_b5c88 && [1/2] Building CXX object CMakeFiles/cmTC_b5c88.dir/src.cxx.o
[2/2] Linking CXX executable cmTC_b5c88

Source file was:
int main() { return 0; }

build.log - 6WV53QV2L.txt

CMakeError.log - 99FFXC737.txt

CMakeOutput.log - HMHA288ZK.txt

[jrprice]

If you're using a sparse-checkout, you'll need to add the recently added /cmake directory to your .gito/info/sparse-checkout file and update to get these additional CMake config files.

[shoober420]

@jrprice

I'm on Gentoo, using the llvm-14.0.0.9999 ebuild. I don't even know where the "sparse-checkout" file is on my system. I don't seem to have a ".git" directory in my home directory or llvm directory.

[Mystro256]

Yeah I get this issue too. The checkout is fine, the files in /cmake/modules exist, but there's some sort of cmake related bug that isn't too clear.

[h-vetinari]

Hitting this issue in conda-forge (https://github.com/conda-forge/llvmdev-feedstock/pull/143), where we have until now built llvm (the subproject) only from the llvm sources (and not llvm-project as a whole).

Do I understand correctly that that's not possible anymore? Is that intentional? Should the cmake/ folder perhaps be part of the produced subproject sources?

From the POV of our build it's a regression (though we'll survive if it is declared unsupported), hence tagging @tstellar, as I cannot set milestones.

[karineek]

I also got this issue. I checked recent changes in cmake/Modules/ folder. This commit looks related to it: https://github.com/llvm/llvm-project/commit/44e3365775101fec3fd355eda339282258d74415. I first tried to upgrade my Cmake from 3.16.3 (ubuntu-20 default) to 3.22.2 (newest), but this didn't solve the issue.

I used its parent commit (git checkout b99833dc) to build llvm with coverage, and it seems to work (after turning off LLVM_INCLUDE_BENCHMARKS flag). It builds version 14.0.0.

[aaronpuchert]

I also got this issue. I checked recent changes in cmake/Modules/ folder. This commit looks related to it: 44e3365775101fec3fd355eda339282258d74415.

Seems likely. @Ericson2314, why put these files in cmake instead of llvm/cmake? Are there projects that don't have access to the latter?

@tstellar, if this stands I think we'll need to provide a tarball for this cmake top-level directory as well, otherwise none of the subproject tarballs are usable anymore.

[aaronpuchert]

@Ericson2314, why put these files in cmake instead of llvm/cmake?

Actually the readme answers that, but I'm not sure I understand:

The advantage of not putting these modules in an existing location like llvm/cmake/modules is two-fold:

• Since they are not installed, we don't have to worry about any out-of-tree downstream usage, and thus there is no need for stability.

Fair enough, but who is forcing us to install all files in llvm/cmake/modules? The installation already excludes GetHostTriple.cmake.

• Since they are available as part of the source at build-time, we don't have to do the usual stand-alone vs combined-build dances, avoiding much complexity.

They are not part of e.g. the llvm subproject tarball though. That used to be self-contained prior to LLVM 14.

[tstellar]

@aaronpuchert This solution exists in the main branch: https://reviews.llvm.org/D118481 Is this OK or would you prefer to have a separate cmake tarball?

[aaronpuchert]

@tstellar, thanks for pointing that out. I guess it came to late for rc1? I've worked with a separate tarball for now, but this should also work, perhaps even without changes on my side. It would give me multiple copies of that directory, but that should be bearable since it's pretty small.

[h-vetinari]

This solution exists in the main branch: https://reviews.llvm.org/D118481

That's great news, thanks! Does 32a0482a65b8 need to be marked for backport in a separate issue, or is this one enough?

[tstellar]

/cherry-pick https://github.com/llvm/llvm-project/commit/32a0482a65b86cf0d84ea7e784cca9852df5c67c

[llvmbot]

Failed to cherry-pick: https://github.com/llvm/llvm-project/commit/32a0482a65b86cf0d84ea7e784cca9852df5c67c

https://github.com/llvm/llvm-project/actions/runs/1913260431

Please manually backport the fix and push it to your github fork. Once this is done, please add a comment like this:

/branch <user>/<repo>/<branch>

[Ericson2314]

Glad /cmake is going in the tarballs now.

[h-vetinari]

Please manually backport the fix and push it to your github fork. Once this is done, please add a comment like this:

/branch h-vetinari/llvm-project/backport_D118481

[llvmbot]

/pull-request llvmbot/llvm-project#110

[h-vetinari]

@tstellar the backport cherry-pick is ready here (leaving a comment since it missed rc2 despite being milestoned, hoping it can get into rc3).

[tstellar]

Merged: 1e3d5ccab6b616baa91448462951b43f28bd5601

[h-vetinari]

This still seems to be an issue with rc4:

CMake Error at cmake/modules/CMakeLists.txt:1 (include):
  include could not find requested file:

    ExtendPath

CMake Error at cmake/modules/CMakeLists.txt:3 (include):
  include could not find requested file:

    FindPrefixFromConfig

CMake Error at cmake/modules/CMakeLists.txt:113 (find_prefix_from_config):
  Unknown CMake command "find_prefix_from_config".

[h-vetinari]

I'm hitting this with both llvm and compiler-rt, so the fix didn't seem to take effect properly... Can we reopen this issue, or should I raise a new one?

[tstellar]

@h-vetinari Can you provide instructions for reproducing this?

[mati865]

@tstellar one of the easiest examples is building compiler-rt on Linux or Windows (tested with mingw-w64):

$ tar -xf compiler-rt-14.0.0rc4.src.tar.xz
$ cd compiler-rt-14.0.0rc4.src/
$ mkdir build && cd build/
$ cmake -G Ninja -DCMAKE_BUILD_TYPE=Release ..
-- The C compiler identification is Clang 12.0.0
-- The CXX compiler identification is Clang 12.0.0
-- The ASM compiler identification is Clang
-- Found assembler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:31 (include):
  include could not find load file:

    SetPlatformToolchainTools

CMake Error at cmake/base-config-ix.cmake:9 (include):
  include could not find load file:

    ExtendPath
Call Stack (most recent call first):
  CMakeLists.txt:32 (include)

-- Looking for unwind.h
-- Looking for unwind.h - found
-- Looking for rpc/xdr.h
-- Looking for rpc/xdr.h - found
CMake Error at cmake/base-config-ix.cmake:108 (extend_path):
  Unknown CMake command "extend_path".
Call Stack (most recent call first):
  CMakeLists.txt:32 (include)

-- Configuring incomplete, errors occurred!
See also "/tmp/compiler-rt-14.0.0rc4.src/build/CMakeFiles/CMakeOutput.log".
See also "/tmp/compiler-rt-14.0.0rc4.src/build/CMakeFiles/CMakeError.log".

EDIT: Updated the steps to be more precise.

[h-vetinari]

Yup, something like @mati865's example is similar to what we're doing (i.e. quite vanilla).

[h-vetinari]

Looking some more at https://github.com/llvm/llvm-project/commit/1e3d5ccab6b616baa91448462951b43f28bd5601, I don't think the following works as intended:

        git archive --prefix=$proj-$release$rc.src/ -o $target_archive_file.tmp $tree_id .
        # Get relative path to top-level cmake directory to be packaged
        # alongside the project. Append that path to the tarball.
        cmake_rel_path=$(realpath --relative-to=. $llvm_src_dir/cmake)
        tar --append -f $target_archive_file.tmp $cmake_rel_path

In particular, I don't think that tar is able to --append a folder. Did you test this somehow @kwk?

AFAICT from the documentation of git archive, it should be possible to add several paths, e.g. something like

        pushd $llvm_src_dir/$proj
        git archive --prefix=$proj-$release$rc.src/ $tree_id . $llvm_src_dir/cmake | xz >$target_dir/$(template_file $proj)

[h-vetinari]

Could someone reopen this issue please?

I'd prefer to keep the context of the discussion so far, but if not, I'll just have to open a new one...

[aaronpuchert]

Looking some more at 1e3d5cc, I don't think the following works as intended:

        git archive --prefix=$proj-$release$rc.src/ -o $target_archive_file.tmp $tree_id .
        # Get relative path to top-level cmake directory to be packaged
        # alongside the project. Append that path to the tarball.
        cmake_rel_path=$(realpath --relative-to=. $llvm_src_dir/cmake)
        tar --append -f $target_archive_file.tmp $cmake_rel_path

Another issue is that it uses the checked-out source tree instead of taking files from tag/commit passed as argument.

AFAICT from the documentation of git archive, it should be possible to add several paths, e.g. something like

        pushd $llvm_src_dir/$proj
        git archive --prefix=$proj-$release$rc.src/ $tree_id . $llvm_src_dir/cmake | xz >$target_dir/$(template_file $proj)

But would it appear in the right place then?

How about using git archive to package up the cmake/ directory with the correct prefix, and then use tar --concatenate to append it to the other archives?

[aaronpuchert]

Posted D121972, feel free to test it.

[kwk]

Looking some more at 1e3d5cc, I don't think the following works as intended:

        git archive --prefix=$proj-$release$rc.src/ -o $target_archive_file.tmp $tree_id .
        # Get relative path to top-level cmake directory to be packaged
        # alongside the project. Append that path to the tarball.
        cmake_rel_path=$(realpath --relative-to=. $llvm_src_dir/cmake)
        tar --append -f $target_archive_file.tmp $cmake_rel_path

In particular, I don't think that tar is able to --append a folder. Did you test this somehow @kwk?

I tested it and it currently runs every day. Take any tar ball in this release (https://github.com/kwk/llvm-daily-fedora-rpms/releases) to see that there's a cmake top-level directory.

AFAICT from the documentation of git archive, it should be possible to add several paths, e.g. something like

        pushd $llvm_src_dir/$proj
        git archive --prefix=$proj-$release$rc.src/ $tree_id . $llvm_src_dir/cmake | xz >$target_dir/$(template_file $proj)

I don't know why I didn't see this or if I didn't test this. It sounds like a good idea.

[kwk]

Another issue is that it uses the checked-out source tree instead of taking files from tag/commit passed as argument.

That indeed is an issue.

[aaronpuchert]

@tstellar, need a backport for this?

[h-vetinari]

Thanks everyone! I'd love to test this, but it's not so easy (I'd have to create the tarballs in CI per platform and I'd have to work around a couple thing). The best thing would be another RC obviously, but I'm guessing that's not in the cards anymore. In any case, I'm (unsurprisingly) in favour of this being backported.

[aaronpuchert]

/cherry-pick 3a33664e8838e8b77acd1bbb13b1cf5e580a1077

[llvmbot]

/branch llvmbot/llvm-project/issue53281

[llvmbot]

/pull-request llvmbot/llvm-project#130

[tstellar]

Merged: 23d08271a4b2

[uaex]

CMake Error at cmake/modules/CMakeLists.txt:1 (include):
  include could not find requested file:

    ExtendPath

CMake Error at cmake/modules/CMakeLists.txt:3 (include):
  include could not find requested file:

    FindPrefixFromConfig

CMake Error at cmake/modules/CMakeLists.txt:113 (find_prefix_from_config):
  Unknown CMake command "find_prefix_from_config".

Configuring incomplete, errors occurred!

I'm puzzled about this issue. It seems that ExtendPath.cmake and FindPrefixFromConfig.cmake does not exist in repository, and the solution does not resolve it. 😞

[aaronpuchert]

It seems that ExtendPath.cmake and FindPrefixFromConfig.cmake does not exist in repository, and the solution does not resolve it.

They definitely exist in the repository: in cmake/Modules/ExtendPath.cmake and cmake/Modules/FindPrefixFromConfig.cmake. They weren't packaged into the subproject tarballs with the release candidates, but this is fixed with 14.0.0.

[SupervisedThinking]

still missing / wrong packed in https://github.com/llvm/llvm-project/releases/download/llvmorg-14.0.0/llvm-14.0.0.src.tar.xz

[heitbaum]

still missing / wrong packed in https://github.com/llvm/llvm-project/releases/download/llvmorg-14.0.0/llvm-14.0.0.src.tar.xz

I included the missing in the LE PR. Assuming that 14.0.1 should be right - https://github.com/heitbaum/LibreELEC.tv/commit/138e929c4a7c0d4d9d1d56a9e9a1f24504fa4ad2

Also - issues with the make options not overriding - but I haven’t had time to look at that further. // why … https://cmake.org/cmake/help/latest/manual/cmake.1.html // https://stackoverflow.com/questions/31037882/whats-the-cmake-syntax-to-set-and-use-variables

[aaronpuchert]

still missing / wrong packed in https://github.com/llvm/llvm-project/releases/download/llvmorg-14.0.0/llvm-14.0.0.src.tar.xz

Can you please run and compare:

> tar --list -f llvm-14.0.0.src.tar.xz | tail
cmake/
cmake/Modules/
cmake/Modules/EnableLanguageNolink.cmake
cmake/Modules/ExtendPath.cmake
cmake/Modules/FindPrefixFromConfig.cmake
cmake/Modules/HandleCompilerRT.cmake
cmake/Modules/HandleOutOfTreeLLVM.cmake
cmake/Modules/LLVMCheckCompilerLinkerFlag.cmake
cmake/Modules/SetPlatformToolchainTools.cmake
cmake/README.rst

For me both ExtendPath and FindPrefixFromConfig are there.

[SupervisedThinking]

as I said they are wrong packed:

They were added to the root directory and not where they belong:

Someone also mixed up ~/cmake/Modules & ~/cmake/modules so those specific files are in a wrong top folder and the folder name is wrong too. (IMHO)

[aaronpuchert]

These are two different folders in the repository: cmake/modules and llvm/cmake/modules. The tarball simply reflects that. Now I'm not sure we need that separate directory, but it was decided to have it to separate files that should be installed from files that shouldn't. (See my earlier comment.)

As for modules vs. Modules, CMake doesn't care. Not sure why we went with different capitalization, but that ship sailed a long time ago:

cmake/Modules
clang/cmake/modules
compiler-rt/cmake/Modules
flang/cmake/modules
libc/cmake/modules
libcxxabi/cmake/Modules
libcxx/cmake/Modules
libunwind/cmake/Modules
lldb/cmake/modules
lld/cmake/modules
llvm/cmake/modules
mlir/cmake/modules

This bug here is simply a packaging issue. If you think we shouldn't have that top-level cmake directory, better open a separate bug. But read that README first.

[sbourdeauducq]

Bug is still present in release 14.0.3.

[aaronpuchert]

Could you be a bit more specific what your setup is? The subproject tarballs contain the necessary files now.

[SupervisedThinking]

@aaronpuchert I guess a 14.0.4 will be fixed then? Or did you re-release the 14.0.3 files with https://github.com/PBHDK/llvm-project/commit/4959d294267ad5b073582f1b00a6c77665b8b863

[aaronpuchert]

The bug is closed because by our understanding the issue has been fixed with 14.0.0, and only release candidates were affected. So what is the issue that you're seeing? The files are there, so somehow the build that you're doing doesn't pick them up.

[sbourdeauducq]

Well, just open the 14.0.3 release tarball (https://github.com/llvm/llvm-project/releases/download/llvmorg-14.0.3/llvm-14.0.3.src.tar.xz). There is a cmake folder at the top level which shouldn't be there in addition to the legitimate llvm-14.0.3.src folder, with consequences as described in this Issue. Other tarballs are also affected.

[heitbaum]

Using llvm-project (14) source now versus llvm (13), this addresses the change of packaging.

• https://github.com/LibreELEC/LibreELEC.tv/pull/6492

[tstellar]

Here is a proposed change to remove the cmake bundling from each tarball and package it separately: https://reviews.llvm.org/D125798 To me this seems like the best option for now. It will get us back into our original state with the tarballs, and then after that we can discuss other improvements like the directory names, etc.

[heitbaum]

Here is a proposed change to remove the cmake bundling from each tarball and package it separately: https://reviews.llvm.org/D125798 To me this seems like the best option for now. It will get us back into our original state with the tarballs, and then after that we can discuss other improvements like the directory names, etc.

As a packaging manager - I would ask that each .tar.gz file can discretely be built in its directory, without the .. syntax.

Whilst the updated package.mk file in LibreELEC uses the override of llvm/CMakeLists.txt - I would have preferred an CMakeLists.txt in the root directory which would have called llvm/CMakeLists.txt.

I would support NOT having 2 parent directories in the tar.gz file as llvm-14 is being shipped. If this is the need - then suggest moving both llvm (remove the version) and cmake as subdirectories in the tar.gz:llvm-14.0.3/ directory.

[aaronpuchert]

There is a cmake folder at the top level which shouldn't be there in addition to the legitimate llvm-14.0.3.src folder, with consequences as described in this Issue.

The issue described here stems from the cmake directory not being there, because the files simply didn't exist. With the directory they should be in ../cmake/Modules as in the git tree and thus should be found.

I would support NOT having 2 parent directories in the tar.gz file as llvm-14 is being shipped. If this is the need - then suggest moving both llvm (remove the version) and cmake as subdirectories in the tar.gz:llvm-14.0.3/ directory.

Isn't that #54716 instead of this issue?