search

easybuilders / easybuild-easyblocks

Collection of easyblocks that implement support for building and installing software with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
106 stars 285 forks source link

enhance `LLVM` easyblock for compilation of clang/flang + other llvm-projects #3373

Open Crivella opened 4 months ago

Crivella commented 4 months ago

This EB is a modified version of the current Clang one (whether to keep them separate, having a simpler EB for Clang-only, or rename Clang to LLVMcore or viceversa is up for debate).

Requires testing of:

It enables building most of the main LLVM projects. What is still missing is/has not been tested:

The current EB improves on the Clang one by:

Currently/WIP:

Notes:

Possible TODOs:

boegel commented 4 months ago

I haven't looked at this in detail, but I'm not a big fan of "forking" an existing easyblock, and then updating it. There seems to be a lot of overlap still with the custom easyblock for Clang.

I do understand that the other option (updating the existing custom easyblock for Clang to introduce updates/enhancements) is painful too though, partially since there's a lot of history there.

@Crivella How difficult would it be to integrate the functionality you need in the existing easyblock for Clang (or LLVM, I'm not even sure which one is most relevant here, but I guess it's the Clang one).

Crivella commented 4 months ago

@boegel I will try to work on it as soon as I finish all the current tests (there is still something to iron out on non shared builds, and understanding if the test failures should be addressed or can be ignored). I went this route originally as I was working both on a standalone toolchain for building the single components separately and this one (and also for ease of testing features out). I will have to check how compatible the way the bootstrap was being done in the previous build with respect to this one, but it should be doable.

I do not know if it would still make sense to keep the Clang naming scheme as this would end up including also Flang and other optimization tools usable also with other frontends.

ocaisa commented 4 months ago

@boegel Isn't it a lot to ask to merge these easyblocks? The reality is that we already have two other forks for LLVM and AOMP, so the precedent already exists for taking a new/alternative path.

Crivella commented 4 months ago

@boegel I've been looking back at the 2 EB side by side and while not impossible to merge them, i think the most feasible approach would be the QE route of calling a different EB based on the version of the software. While a lot of pieces have been taken from the Clang EB, here I've taken a fundamentally different approach for a few options and for the bootstrap build which would require deprecating options from the old EB if a new version is used. Concerning the bootstrap build, in this EB, not all tools are being built at all stages of the build (only the minimally required one) and different options are being passed at different stages which would require rewriting half of the Clang EB to merge.

There is also the fact that for the Clang EB, flang is just an optional part of the build to be manually specified through the llvm_projects extra option. The idea behind this EB is not to build just Clang but all the tooling required to have an LLVM toolchain in place which would require both clang and flang to always be present

ocaisa commented 4 months ago

One thing I don't actually like is the name, clang.py, because it is not just clang, it is LLVM with front end compilers. That is part of the complication, because in reality it should be llvm.py, and clang.py should stop being used. QE-style forking within the relevant easyblocks seems like the only available option.

Crivella commented 4 months ago

As an update, i've also tested now with all tools enabled and build_targets = ['all']. The compilation + test_suite was successful with only a slight increase in errors (with respect to the previous bootstrap+full_llvm build with default build target):

It should be noted that before 34341 (28.84%) were flagged as unsupported while now only 4588 (3.85%) which should be due to the experimental archs

Crivella commented 4 months ago

UPDATE

Testing with openmp offloading on VEGA frontend node:

With:

build_targets = ['X86', 'NVPTX']
cuda_compute_capabilities = ['8.0']

I encountered extra errors related to libomptarget:

As a sanitycheck I also recompiled on my WS by allowing libomptarget to be compiled for x86 (before i had it enabled only for nvptx and amd) and i get a similar number of test errors (268)

Beside that i tried to compile a simple openmp application and the GPU offloading seems to be working correctly (verified the GPU is being used from nvtop)

Crivella commented 4 months ago

UPDATE

Testing with openmp offloading on VEGA frontend node:

With:

build_targets = ['X86', 'NVPTX']
cuda_compute_capabilities = ['8.0']

I encountered extra errors related to libomptarget:

As a sanitycheck I also recompiled on my WS by allowing libomptarget to be compiled for x86 (before i had it enabled only for nvptx and amd) and i get a similar number of test errors (268)

Beside that i tried to compile a simple openmp application and the GPU offloading seems to be working correctly (verified the GPU is being used from nvtop)

Thyre commented 4 months ago

I've tried to build LLVM using your easyblock and easyconfig PRs on a fresh devel installation. I had some issues with Python, where the tests reported that Python itself was not found even though it was installed. After disabling the Python parts I had no trouble building both LLVMcore variants.

I haven't looked to deeply into which and how many tests failed though. I built everything on my local workstation (Intel Core i7-12700, AMD Radeon RX7700XT).

Crivella commented 4 months ago

@Thyre

Thanks for the report, will look into it The error was related to the python_bindings = True setting in the EC files?

Thyre commented 4 months ago

@Thyre

Thanks for the report, will look into it The error was related to the python_bindings = True setting in the EC files?

Yes, exactly. When setting it to False, everything worked fine. The steps to reproduce were basically:

  1. Clone the EB repos (with your PRs) (instructions taken from here)
  2. Run the following command: eb --robot LLVMcore-18.1.7.eb LLVMcore-18.1.7-GCCcore-13.3.0.eb --parallel 8
Crivella commented 4 months ago

I had some issues with Python, where the tests reported that Python itself was not found even though it was installed.

I have an idea on what might be happening. Was the error during the test_step or the sanity_check_step.

In case it is the latter it is very likely that the reason would be that I had another python available and capable of working on my WS during the build which was causing the sanity check to pass even if python was not in the normal dependencies instead of the build ones.

Let me know if this is the case, in the meanwhile i will fix the EC files

Crivella commented 4 months ago

UPDATE:

after some discussion in the 2024/07/17 Easybuild Conference Call, it was decided to replace the EB_LLVM easyblock with the new one from this PR.

To this end, the following has been added:

The following tests have been manually performed for sanity check:

Crivella commented 2 months ago

Following a discussion in slack about the deprecation warning about GCC_INSTALL_PREFIX switched to an error for LLVM 19 (see also https://github.com/llvm/llvm-project/pull/85891):

Crivella commented 1 month ago

Changed runtimes_cmake_args into a list to allow modifications from multiple parts of the code (will be checked and added before each configure call.

Ensure correct python is detected also during the RUNTIMES CMake call

This python discussion overlaps with the discussion in:

Crivella commented 1 month ago

As an update everything should work now with LLVM 19.1.1 (manually tested with EC from https://github.com/easybuilders/easybuild-easyconfigs/pull/21611)

The problem related to https://github.com/llvm/llvm-project/issues/111667 should be fixed by 7151ec2 adding --unwindlib=none to CMAKE_EXE_LINKER_FLAGS during the runtimes compilation

WIP:

Micket commented 1 month ago

Related? https://github.com/llvm/llvm-project/issues/90333

Crivella commented 1 month ago

Related? llvm/llvm-project#90333

Seems to be related, the offload CMake does not seem to be properly configuring OPENMP_TEST_OPENMP_FLAGS. Tried to manually patch the -fopenmp but then this results in a omp.h not found, so I guess I will also have to patch the location of the header during the test

Crivella commented 1 month ago

@Thyre In reference to:

Modularized functionalities related to detecting/setting/sanity_checking gcc_prefix in tre functions (https://github.com/easybuilders/easybuild-easyblocks/pull/3373/commits/997a689f041ace9b9e79a1ee6b0ac34aa237a185):

Tested with EC from:

Crivella commented 1 month ago

@Micket The solution was slightly more involved but atleast on x86_64 was able to make a patch file (needs testing on other archs and gpus). Opened an issue on llvm, hope it will get picked up and maybe fix the problem at the source in a more robust way

Thyre commented 1 month ago

@Thyre In reference to:

* [Enhance AOCC EasyBlock to correctly pass GCC toolchain and compiler driver #3480 (comment)](https://github.com/easybuilders/easybuild-easyblocks/pull/3480#issuecomment-2410589255)

Modularized functionalities related to detecting/setting/sanity_checking gcc_prefix in tre functions (997a689):

* get_gcc_prefix

* create_compiler_config_file

* sanity_check_gcc_prefix

Tested with EC from:

* [{compiler}[SYSTEM,GCCcore/13.3.0] LLVM 19.1.1 easybuild-easyconfigs#21611](https://github.com/easybuilders/easybuild-easyconfigs/pull/21611)

Great, thanks a lot! Once this PR gets merged, we should probably cleanup the AOCC EasyBlock and maybe include these checks and/or settings in the other LLVM-based compilers.

Thyre commented 1 month ago

As a (very early) heads-up: LLVM 20 will include the rename of flang-new to flang (https://github.com/llvm/llvm-project/pull/110023). A symlink to flang-new still exists.