easybuilders / easybuild-easyblocks

Collection of easyblocks that implement support for building and installing software with EasyBuild.
https://easybuild.io
GNU General Public License v2.0
104 stars 284 forks source link

clangversion string style fails sanity check in latest AOCC #3429

Open matthewabrown opened 2 weeks ago

matthewabrown commented 2 weeks ago

https://github.com/easybuilders/easybuild-easyblocks/blob/cc5b95dab1aabae2a64360c492a4f068f8eab267/easybuild/easyblocks/a/aocc.py#L150

I'm creating a new easyconfig AOCC-4.2.0-GCCcore-13.2.0.eb. I use the latest 4.0.0 easyconfig but

The user supplied dotted decimal (16.0.3) is reused during sanity check but the AOCC installer uses major version (16) only.

# EasyConfigs have a clear pattern of using dotted decimals
$ grep clang *
AOCC-3.1.0-GCCcore-10.2.0.eb:clangversion = '12.0.0'
AOCC-3.1.0-GCCcore-10.3.0.eb:clangversion = '12.0.0'
AOCC-3.1.0-GCCcore-11.2.0.eb:clangversion = '12.0.0'
AOCC-3.2.0-GCCcore-11.2.0.eb:clangversion = '13.0.0'
AOCC-3.2.0-GCCcore-11.3.0.eb:clangversion = '13.0.0'
AOCC-4.0.0-GCCcore-11.3.0.eb:clangversion = '14.0.6'
AOCC-4.0.0-GCCcore-12.2.0.eb:clangversion = '14.0.6'
AOCC-4.0.0-GCCcore-12.3.0.eb:clangversion = '14.0.6'
AOCC-4.2.0-GCCcore-13.2.0.eb:clangversion = '16.0.3'

Installation failed during sanity checking:

== postprocessing...
== sanity checking...
== ... (took 1 secs)
== FAILED: Installation ended unsuccessfully (build directory: /dev/shm/brownm12/build/AOCC/4.2.0/GCCcore-13.2.0): build failed (first 300 chars): Sanity check failed: no file found at 'lib/clang/16.0.3/include/omp.h' in /apps/easybuild/software/owl-genoa/AOCC/4.2.0-GCCcore-13.2.0
no file found at 'lib/clang/16.0.3/include/stddef.h' in /apps/easybuild/software/owl-genoa/AOCC/4.2.0-GCCcore-13.2.0
no (non-empty) directory found at 'lib/clang/16. (took 52 secs)

In the build and install directories, I can confirm that I have the header files such as this one

$ find . -name "omp.h"
./lib/clang/16/include/omp.h

Perhaps this is different behavior from previous versions of AOCC. The easiest fix might be to add some logic to the sanity check in the easyblock to provide version strings that match the expected format.

matthewabrown commented 2 weeks ago

For what it's worth, if use the short version

clangversion = '16'

in the easyconfig, the installation succeeds with no errors. But not using the dotted decimal seems contrary to all other comments and logic around clangversion.

Thyre commented 2 weeks ago

Looking at my local (non-EasyBuild) LLVM/Clang installations, the same thing can be seen:

$ ls */lib/clang
12.0.1/lib/clang:
12.0.1

13.0.1/lib/clang:
13.0.1

14.0.6/lib/clang:
14.0.6

15.0.7/lib/clang:
15.0.7

16.0.6/lib/clang:
16

17.0.6/lib/clang:
17

18.1.8/lib/clang:
18

19.1.0-rc3/lib/clang:
19

WIth LLVM 16, they switched to just using the major version as the folder name. AOCC basically just copied the LLVM/Clang behavior.

Thyre commented 2 weeks ago

We could probably add some handling for this, like done in the clang.py EasyBlock:

diff --git a/easybuild/easyblocks/a/aocc.py b/easybuild/easyblocks/a/aocc.py
index ff33673af..71156fa4a 100644
--- a/easybuild/easyblocks/a/aocc.py
+++ b/easybuild/easyblocks/a/aocc.py
@@ -138,6 +138,12 @@ class EB_AOCC(PackedBinary):

     def sanity_check_step(self):
         """Custom sanity check for AOCC, based on sanity check for Clang."""
+
+        # Clang v16+ only use the major version number for the resource dir
+        resdir_version = self.clangversion
+        if self.clangversion >= '16':
+            resdir_version = LooseVersion(self.clangversion).version[0]
+
         shlib_ext = get_shared_lib_ext()
         custom_paths = {
             'files': [
@@ -147,7 +153,7 @@ class EB_AOCC(PackedBinary):
                 'lib/clang/%s/include/stddef.h' % self.clangversion, 'lib/libclang.%s' % shlib_ext,
                 'lib/libomp.%s' % shlib_ext,
             ],
-            'dirs': ['include/llvm', 'lib/clang/%s/lib' % self.clangversion, 'lib32'],
+            'dirs': ['include/llvm', 'lib/clang/%s/lib' % resdir_version, 'lib32'],
         }

         custom_commands = [

At the same time, I would suggest updating the version list in _aocc_guess_clang_version.