search

UCL-ARC / hpc-spack

Solutions - HPC's Spack config
MIT License
1 stars 2 forks source link

CASTEP 23.1.1 and CASTEP 24.1 #59

Open heatherkellyucl opened 4 months ago

heatherkellyucl commented 4 months ago

The existing recipe covers CASTEP 19.1.1.rc2 and 21.11.

I have added the checksum for 23.1.1 in repos/dev/packages/castep/package.py

Need to check if there are any flags or variants needed for this version, if it compiles and if it works.

heatherkellyucl commented 2 months ago

No difference. (I am guessing that the difference in failure point might be that spack was building in parallel. It also automatically adds some more cmake flags).

Confirmed also that this is exactly the same:

cmake -B ../castep_2023.1.1/build -DWITH_MPI=ON -DWITH_LIBXC=ON -DCMAKE_INSTALL_PREFIX=../castep_2023.1.1
cmake --build ../castep_2023.1.1/build -t all
heatherkellyucl commented 2 months ago

The default target is all and for an MPI build:

ifeq ($(COMMS_ARCH),mpi)
all:     castep utilities

The buildinfo for utilities looks like what is failing. Check the simplest case if we only make castep and also don't include libxc. Then add libxc back in.

(Might want to add what some of the other recipes do, and do a serial build next to the parallel one so that tools is built).

heatherkellyucl commented 2 months ago

Still fails when only building castep.

[ 76%] Linking Fortran static library ../../lib/libdispersion.a
[ 76%] Built target dispersion
[ 76%] Generating buildinfo_data.f90
fatal: Not a git repository (or any parent up to mount point /lustre)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
CMake Error at /home/ccspapp/Scratch/castep/castep-castep-65582dd54bf3/cmake/buildinfo_script.cmake:28 (string
):
  string sub-command STRIP requires two arguments.

gmake[3]: *** [Source/CMakeFiles/CASTEP_BUILDINFO_DATA_OBJ.dir/build.make:75: buildinfo/buildinfo_data.f90] Er
ror 1
gmake[2]: *** [CMakeFiles/Makefile2:848: Source/CMakeFiles/CASTEP_BUILDINFO_DATA_OBJ.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:621: CMakeFiles/castep.dir/rule] Error 2
gmake: *** [Makefile:199: castep] Error 2
heatherkellyucl commented 1 month ago

Kane suggested that bin/mkbuildinfo.pl needed to be run before the build step, and doing this works:

cmake -B ../castep_2023.1.1/build -DWITH_MPI=ON -DCMAKE_INSTALL_PREFIX=../castep_2023.1.1
COMMS_ARCH=mpi ./bin/mkbuildinfo.pl ../castep_2023.1.1/build/buildinfo/buildinfo_data.f90
cmake --build ../castep_2023.1.1/build -t castep

It can then find buildinfo_data.f90 and build it.

[ 76%] Building Fortran object Source/CMakeFiles/CASTEP_BUILDINFO_DATA_OBJ.dir/__/buildinfo/buildinfo_data.f90
.o
[ 76%] Built target CASTEP_BUILDINFO_DATA_OBJ

and the build proceeds to the end.

[100%] Linking Fortran static library ../../lib/libfunctional.a
[100%] Built target functional
[100%] Building Fortran object Source/CMakeFiles/castep.mpi.dir/main.f90.o
[100%] Linking Fortran executable ../bin/castep.mpi
[100%] Built target castep.mpi
[100%] Built target castep
heatherkellyucl commented 1 month ago

Have worked out how to fix cmake/buildinfo_script.cmake and so this can be patched by Spack instead of trying to insert another step into the build method.

(It's the regex).

heatherkellyucl commented 1 month ago 
git diff cmake/buildinfo_script.cmake.bak cmake/buildinfo_script.cmake > castep_2023.1.1_buildinfo.patch

The initial patch is at https://github.com/UCL-ARC/hpc-spack/blob/0.22/repos/dev/packages/castep/castep_2023.1.1_buildinfo.patch and the package.py is updated to use it with 23.1.1 and cmake, now need to test.

heatherkellyucl commented 1 month ago

With the libxc build in Spack, something seems to be generating the lower-case version of not a git repository too, so I am trying adding a TOLOWER on the output before running the regex.

Also if you change a patch and Spack complains, spack clean -m and then full reconcretisation should sort it:

==> Error: castep-23.1.1-zns3oxrsrefdfcxsl7q4vab6wh3mrmss: sha256 checksum failed for /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/repos/dev/packages/castep/castep_2023.1.1_buildinfo.patch
    Expected a02656c8cd112d68d6bb9f304e69dc87d3df6d0609afea7d94240b19d1573488 but got 6bd50441953247ceb38fbdbd2bd644f9efa53d9d0d36bccd5f4d534a1fa59ed7 Patch may have changed since concretization.
==> Error: castep-23.1.1-zns3oxrsrefdfcxsl7q4vab6wh3mrmss: Package was not installed
==> Error: Installation request failed.  Refer to reported errors for failing package(s).

If you only spack clean -m and a normal concretize then you get

==> Error: castep-23.1.1-zns3oxrsrefdfcxsl7q4vab6wh3mrmss: Couldn't find patch for package ucl.arc.hpc.dev.castep with sha256: a02656c8cd112d68d6bb9f304e69dc87d3df6d0609afea7d94240b19d1573488. This usually means the patch was modified or removed. To fix this, either reconcretize or use the original package repository
==> Error: castep-23.1.1-zns3oxrsrefdfcxsl7q4vab6wh3mrmss: Package was not installed
==> Error: Installation request failed.  Refer to reported errors for failing package(s).

With both it will have cleared out all the previous patch checksums.

(spacksite: hk-initial-stack) [ccspapp@build01 hpc-spack]$ spack clean -m
==> Removing cached information on repositories
(spacksite: hk-initial-stack) [ccspapp@build01 hpc-spack]$ spack -e myriad concretize -f
...
==> Installing castep-23.1.1-cdd53ikkgvpcybct2i6fpaen2ox5aqml [100/103]
==> No binary for castep-23.1.1-cdd53ikkgvpcybct2i6fpaen2ox5aqml found: installing from source
==> Using cached archive: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/cache/_source-cache/archive/8d/8d922c641c99fc6e4f5b4f7f2478abab897065850e454fb4968154ddbb566388.tar.bz2
==> Applied patch /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/repos/dev/packages/castep/castep_2023.1.1_buildinfo.patch
==> castep: Executing phase: 'cmake'
==> castep: Executing phase: 'build'
heatherkellyucl commented 1 month ago

Ok, the build succeeded. The install did not. Progress!

==> Installing castep-23.1.1-cdd53ikkgvpcybct2i6fpaen2ox5aqml [100/103]
==> No binary for castep-23.1.1-cdd53ikkgvpcybct2i6fpaen2ox5aqml found: installing from source
==> Using cached archive: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/cache/_source-ca
che/archive/8d/8d922c641c99fc6e4f5b4f7f2478abab897065850e454fb4968154ddbb566388.tar.bz2
==> Applied patch /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/repos/dev/packages/castep/castep_2023.1
.1_buildinfo.patch
==> castep: Executing phase: 'cmake'
==> castep: Executing phase: 'build'
==> castep: Executing phase: 'install'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j6' 'install'

2 errors found in build log:
...
     2896    -- Installing: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_
             placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/_
             _spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-casc
             adelake/gcc-12.3.0/castep-23.1.1-cdd53ikkgvpcybct2i6fpaen2ox5aqml/lib64/libxcf90.a
  >> 2897    CMake Error at LibSource/libxc-5.2.2/cmake_install.cmake:92 (file):
     2898      file INSTALL cannot find
     2899      "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-23
             .1.1-cdd53ikkgvpcybct2i6fpaen2ox5aqml/spack-build-cdd53ik/LibSource/libxc-5.2.2/xc_f03_lib_m.mod
             ":
     2900      No such file or directory.
     2901    Call Stack (most recent call first):
     2902      cmake_install.cmake:47 (include)
     2903
     2904
  >> 2905    make: *** [Makefile:133: install] Error 1

See build log for details:
  /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-23.1.1-cdd53ikkgvpcybct2i6fpaen2ox5aqml/spack-build-out.txt
krefson commented 1 month ago

I've just been alerted to this thread. Sorry you are having so much difficulty with the cmake build - this was new in 23.1 and there is much complexity needed to support the build and test process flexibly and fully.

The first issue you found is simply that older versions of git report "Not a git repo" which was later changed to "not a git repo", and is fixed by a simple change to the regexp ("[Nn]ot a git repo") in cmake/buildinfo_script.cmake - which I will include for 25.1.

[scarf035@cn1052 ~]$ /usr/bin/git --version
git version 1.8.3.1
[scarf035@cn1052 ~]$ /usr/bin/git status --short --untracked=no --porcelain
fatal: Not a git repository (or any parent up to mount point /home/vol05)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
[scarf035@cn1052 ~]$ ~/bin/git --version
git version 2.9.5
[scarf035@cn1052 ~]$ ~/bin/git status --short --untracked=no --porcelain
fatal: Not a git repository (or any parent up to mount point /home/vol05)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).
[scarf035@cn1052 ~]$ module load git
[scarf035@cn1052 ~]$ git --version
git version 2.42.0
[scarf035@cn1052 ~]$ git status --short --untracked=no --porcelain
fatal: not a git repository (or any parent up to mount point /home)
Stopping at filesystem boundary (GIT_DISCOVERY_ACROSS_FILESYSTEM not set).

The other issue is more concerning. The purpose of the "gitinfo.dat" file is an attempt to track the metadata of the CASTEP version across a tarball distribution. If it is missing, that means the source is of unknown provenance, and my inclination is that it is good that such a build fails - certainly in the case of an HPC install! The official release tarball on https://licences.stfc.ac.uk/ has it:

$ tar tf ~/Downloads/CASTEP-23.1.tar.gz | grep gitinfo.dat CASTEP-23.1/gitinfo.dat

so something is amiss or astray with the source. I do not recommend running bin/mkbuildinfo.pl for a cmake build as this is the script for the GNU make build.

We did release an update patch for 23.1 in addition to the official release, which may be downloaded from licenses.stfc.ac.uk. This ought also to fix any issues with the compile flags of the liibraries with gfortran 10 and above, as well as a couple of build issues on ARCHER2 highlighted by Andy Turner.

77c102798890a9d15faa63e9122c4d06 Castep_23.1_build_fixes.diff.gz ab349319881334ad55b49dc9765beb06 CASTEP-23.1.tar.gz

This will still need the patch to the regexp in cmake/buildinfo_script.cmake if your version of git is too old.

The cmake build is more stable in 24.1, which was released six months ago, by the way.

heatherkellyucl commented 1 month ago

@krefson

Oh, castep.org still only has news for 23.1.1 as the most recent release so I didn't go looking if I could get a download for 24.

I have castep-23.1.1.tar.bz2 which untars as castep-castep-65582dd54bf3 and I got from the official sources - possibly in June 2023? (Or at least that's the date on the tarball, I'm assuming I got it a bit later).

krefson commented 1 month ago

Thanks for pointing out the out-of-date news release headline on the licenses site.

65582dd54bf3 is the git hash we tagged as Castep-23.1.1_release. But possibly created "unofficially" by some file transfer/copy? We use the GNU make target "make dist" to create the tarball, which untars into CASTEP-23.1 and importantly, creates "gitinfo.dat" on the fly.

FYI, that gitinfo.dat reads

GIT_HASH=65582dd54
GIT_BRANCH=Castep231_branch
GIT_DATE=Wed May 17 11:09:44 2023 +0100
GIT_MODIFIED=

I would recommend downloading and applying the patch Castep_23.1_build_fixes.diff which also fixes a couple of significant bugs, as well as a compile error with some of the additional tools, which it appears you may also have run into.

heatherkellyucl commented 1 month ago

I've attempted to sign up for the new version so I can get it and the patch - when we got the last one was definitely pre the current site, so it might have been in the transition period when the license signup was being reworked.

heatherkellyucl commented 1 month ago

Alrighty, got 24.1 and the updated 23.1 and fixes.

heatherkellyucl commented 1 month ago

Right, finally getting back to this.

Going to test the outside-Spack build of 23 with the patches first. If I add Spack's git to my modules I should have the same behaviour in both cases.

module use /shared/ucl/apps/spack/0.22/hk-initial-stack/modules/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/var/spack/environments/myriad/linux-rhel7-cascadelake/
module load gcc/12.3.0/gcc-11.2.1
module load cmake/3.27.9/gcc-12.3.0
module load rsync/3.2.7/gcc-12.3.0
module load openblas/0.3.26/gcc-12.3.0 
module load fftw/3.3.10-openmpi/gcc-12.3.0
module load perl/5.38.0/gcc-11.2.1
module load openmpi/4.1.6/gcc-12.3.0 
module load libxc/6.2.2/gcc-12.3.0
module load git/2.42.0/gcc-12.3.0
heatherkellyucl commented 1 month ago 
gunzip -c Castep_23.1_build_fixes.diff.gz > Castep_23.1_build_fixes.diff
patch -p1 < Castep_23.1_build_fixes.diff

Patched things. (Note: Spack can use a gzipped patch from a url as long as you give it the sha256 for the archive as well as the expanded file).

cmake -B ../castep_23.1_install/build -DWITH_MPI=ON -DWITH_LIBXC=ON -DCMAKE_INSTALL_PREFIX=../castep_23.1.1_install
cd ../castep_23.1_install/build/
make
make install

The build completed without any issues, the install still failed at libxc.

[100%] Built target castep.mpi
Install the project...
-- Install configuration: "FAST"
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/include/xc.h
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/include/xc_funcs.h
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/include/xc_funcs_worker.h
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/include/xc_funcs_removed.h
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/include/xc_version.h
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/bin/xc-info
-- Set runtime path of "/home/ccspapp/Scratch/castep/castep_23.1.1_install/bin/xc-info" to ""
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/lib64/libxc.a
-- Up-to-date: /home/ccspapp/Scratch/castep/castep_23.1.1_install/lib64/libxc.a
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/lib64/libxcf03.a
-- Installing: /home/ccspapp/Scratch/castep/castep_23.1.1_install/lib64/libxcf90.a
CMake Error at LibSource/libxc-5.2.2/cmake_install.cmake:92 (file):
  file INSTALL cannot find
  "/home/ccspapp/Scratch/castep/castep_23.1_install/build/LibSource/libxc-5.2.2/xc_f03_lib_m.mod":
  No such file or directory.
Call Stack (most recent call first):
  cmake_install.cmake:47 (include)

make: *** [Makefile:130: install] Error 1

Although I actually want it to use external libxc, so would need to pass it more things to make it do that. And tell Spack to build and use libxc 5.2.2 rather than 6.2.2 for this. [Actually, 5.2.3 - it only fixes one CUDA issue]

heatherkellyucl commented 1 month ago

xc_f03_lib_m.mod does exist in build/mod now, but the install is trying to get it from build/LibSource/libxc-5.2.2.

Line 92 of LibSource/libxc-5.2.2/cmake_install.cmake is indeed trying to install from build/LibSource/libxc-5.2.2/xc_f03_lib_m.mod into "${CMAKE_INSTALL_PREFIX}/include".

I'll have a look at 24 since that fixed more things.

heatherkellyucl commented 1 month ago

Have to work out how to tell it that patches are in the source mirror (or anywhere on the local filesystem) rather than the package repo before I can install any versions via Spack.

==> Error: cannot load package 'castep' from the 'ucl.arc.hpc.dev' repository: FilePatch: Patch file Castep_23.1_build_fixes.diff.gz for package ucl.arc.hpc.dev.castep does not exist.
heatherkellyucl commented 1 month ago

The thing I was missing was that you don't also give the filename when you want it to be a UrlPatch:

    patch(
        f"file://{os.getcwd()}/Castep_23.1_build_fixes.diff.gz",
        sha256="a7860dc6677955d9bc877859666c4e8aa59635723968661674283a3939d7a66b",
        archive_sha256="5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2",
        when="@23.1.1",
    )
heatherkellyucl commented 1 month ago

While Spack is building castep 24.1 with libxc using the existing recipe, I'm having a look at making 23.1.1 use an existing libxc.

While I'm at it, I should also add in python so it isn't looking at the system python2.

module load libxc/5.2.3/gcc-12.3.0
module load python/3.11.6/gcc-12.3.0

-DEXTERNAL_LIBXC_LIBRARY=ON (not recommended!) also needs adding for this. See if cmake then finds it.

heatherkellyucl commented 1 month ago

CASTEP 23.1.1 by hand with external libxc

cmake -B ../castep_23.1_install/build -DWITH_MPI=ON -DWITH_LIBXC=ON -DEXTERNAL_LIBXC_LIBRARY=ON -DCMAKE_INSTALL_PREFIX=../castep_23.1_install
Extralibs:
-- Fox CML: OFF
-- libxc found: /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/libxc-5.2.3-5nhmyelcaign3oojiprkng26oiteomc3/lib/libxc.so
-- Build type: FAST
-- spglib: BUILDING
-- dl_mg: BUILDING
-- dftd3: BUILDING
-- dftd4: BUILDING
-- QUIP: OFF
-- Could NOT find LATEX (missing: LATEX_COMPILER BIBTEX DVIPS)
-- f2py or f90wrap not found, castep_python unavailable
-- Setting parallelism to 4-way MPI with a total of 36
-- Found python as /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/python-3.11.6-jy3jbadp7rsw4six4ivtgk5dtimpkwfx/bin/python3.11

(Can consider the f2py later).

cd ../castep_23.1_install/build/
make

Now the same libxc mod file isn't found by OTFG. It exists in /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/libxc-6.2.2-wpzhhjgxdkufg4lygkxvrhosdijcoozj/include/ and we're setting CMAKE_PREFIX_PATH in our module but not CPATH or INCLUDE_PATH.

[ 25%] Building Fortran object Source/OTFG/CMakeFiles/otfg.dir/xc_kernels.f90.o
/home/ccspapp/Scratch/castep/CASTEP-23.1/Source/OTFG/xc_kernels.f90:6455:9:

 6455 |     use xc_f03_lib_m
      |         1
Fatal Error: Cannot open module file 'xc_f03_lib_m.mod' for reading at (1): No such file or directory
compilation terminated.
make[2]: *** [Source/OTFG/CMakeFiles/otfg.dir/build.make:205: Source/OTFG/CMakeFiles/otfg.dir/xc_kernels.f90.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:985: Source/OTFG/CMakeFiles/otfg.dir/all] Error 2
make: *** [Makefile:166: all] Error 2

CASTEP 24.1 in Spack

Meanwhile, castep 24.1 which built its own libxc completes the build but has the same make install issue with the .mod file locations as I had previously.

==> Installing castep-24.1-tbupnjsvai35jtus65tkfd2eapxy2vox [101/103]
==> No binary for castep-24.1-tbupnjsvai35jtus65tkfd2eapxy2vox found: installing from source
==> Fetching file:///shared/ucl/apps/pkg-store/castep/castep-24.1.tar.gz
==> No patches needed for castep
==> castep: Executing phase: 'cmake'
==> castep: Executing phase: 'build'
==> castep: Executing phase: 'install'
==> Error: ProcessError: Command exited with status 2:
    'make' '-j6' 'install'

2 errors found in build log:

  >> 3373    CMake Error at LibSource/libxc-5.2.3/cmake_install.cmake:92 (file):
     3374      file INSTALL cannot find
     3375      "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-24
             .1-tbupnjsvai35jtus65tkfd2eapxy2vox/spack-build-tbupnjs/LibSource/libxc-5.2.3/xc_f03_lib_m.mod":
     3376      No such file or directory.
     3377    Call Stack (most recent call first):
     3378      cmake_install.cmake:47 (include)
     3379
     3380
  >> 3381    make: *** [Makefile:133: install] Error 1

As with 23, they're all in spack-build-tbupnjs/mod after the make step instead, so this probably needs patching.

heatherkellyucl commented 1 month ago

LibSource/libxc-5.2.3/CMakeLists.txt install parts starts at line 513. In the fortran part there is:

if(ENABLE_FORTRAN)
  install(TARGETS xc xcf03 xcf90
    EXPORT f_interface
    RUNTIME DESTINATION ${CMAKE_INSTALL_BINDIR}
    ARCHIVE DESTINATION ${CMAKE_INSTALL_LIBDIR}
    LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR}
    PUBLIC_HEADER DESTINATION ${CMAKE_INSTALL_INCLUDEDIR})
  install(FILES "${PROJECT_BINARY_DIR}/xc_f03_lib_m.mod"
    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}${NAMESPACE_INSTALL_INCLUDEDIR})
  install(FILES "${PROJECT_BINARY_DIR}/xc_f90_lib_m.mod"
    DESTINATION ${CMAKE_INSTALL_INCLUDEDIR}${NAMESPACE_INSTALL_INCLUDEDIR})
  if(NOT MSVC)
    install(FILES ${PROJECT_BINARY_DIR}/libxcf03.pc
                  ${PROJECT_BINARY_DIR}/libxcf90.pc
      DESTINATION ${CMAKE_INSTALL_LIBDIR}/pkgconfig/)
  endif()
endif()

"${PROJECT_BINARY_DIR}/xc_f03_lib_m.mod" isn't where the mod file is.

heatherkellyucl commented 1 month ago

Looking through the generated build directory's CMakeCache.txt:

//Value Computed by CMake
CASTEP_BINARY_DIR:STATIC=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-24.1-tbupnjsvai35jtus65tkfd2eapxy2vox/spack-build-tbupnjs

//Value Computed by CMake
CASTEP_SOURCE_DIR:STATIC=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-24.1-tbupnjsvai35jtus65tkfd2eapxy2vox/spack-src

...

//Value Computed by CMake
Libxc_BINARY_DIR:STATIC=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-24.1-tbupnjsvai35jtus65tkfd2eapxy2vox/spack-build-tbupnjs/LibSource/libxc-5.2.3

//Value Computed by CMake
Libxc_IS_TOP_LEVEL:STATIC=OFF

//Value Computed by CMake
Libxc_SOURCE_DIR:STATIC=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-24.1-tbupnjsvai35jtus65tkfd2eapxy2vox/spack-src/LibSource/libxc-5.2.3

The mod files are all in $CASTEP_BINARY_DIR/mod.

heatherkellyucl commented 1 month ago

Assuming it is analogous to this:

CMAKE_SOURCE_DIR refers to the top-level source directory that contains a CMakeLists. txt , while PROJECT_SOURCE_DIR refers to the source directory of the most recent project() command

then I think it wants ${CMAKE_BINARY_DIR}/mod and not ${PROJECT_BINARY_DIR}

(If that exists and is usable here).

https://cmake.org/cmake/help/latest/variable/CMAKE_BINARY_DIR.html

The path to the top level of the build tree.

This is the full path to the top level of the current CMake build tree. For an in-source build, this would be the same as CMAKE_SOURCE_DIR.

When run in cmake -P script mode, CMake sets the variables CMAKE_BINARY_DIR, CMAKE_SOURCE_DIR, CMAKE_CURRENT_BINARY_DIR and CMAKE_CURRENT_SOURCE_DIR to the current working directory.

It is going to be safer to refer to $CASTEP_BINARY_DIR/mod, I will have to find out if libxc still knows about that or not.

heatherkellyucl commented 1 month ago

None of the other LibSource do this so it shouldn't be an issue for them.

heatherkellyucl commented 1 month ago

Added a patch, https://github.com/UCL-ARC/hpc-spack/blob/0.22/repos/dev/packages/castep/castep_cmake_libxc.patch - needs to be added to the package.py and tested with 23 and 24.

heatherkellyucl commented 1 month ago

Spack has built and installed CASTEP 24.1! With libxc, built by castep.

Now to see if it can 23.1 too without any additional changes.

heatherkellyucl commented 1 month ago

Got this for 23.1.1 - er, caching?

==> Installing castep-23.1.1-qzeblxq5jdhuwm3hwsr3pby5m3gtma4b [101/104]
==> No binary for castep-23.1.1-qzeblxq5jdhuwm3hwsr3pby5m3gtma4b found: installing from source
==> Fetching file:///shared/ucl/apps/pkg-store/castep/castep-23.1.1.tar.gz
==> Error: FetchError: Manual download is required for castep. Refer to http://castep.org for download instruc
tions.

I did update the checksums, and it is finding the source from our mirror but doesn't want to use it.

heatherkellyucl commented 1 month ago

Oh, maybe it is the versions, it untars as CASTEP-23.1 now and not 23.1.1.

heatherkellyucl commented 1 month ago

No that updated the hash but it still fails:

==> Installing castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2 [100/104]
==> No binary for castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2 found: installing from source
==> Fetching file:///shared/ucl/apps/pkg-store/castep/castep-23.1.tar.gz
==> Error: FetchError: Manual download is required for castep. Refer to http://castep.org for download instructions.
heatherkellyucl commented 1 month ago

It put it into /shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2/CASTEP-23.1.tar.gz and then stopped.

heatherkellyucl commented 1 month ago

Debug: checks for existence in the various caches and mirrors, and with hashes in the names - those are not expected to be found.

Does ==> [2024-09-18-12:41:08.286005] Fetching file:///shared/ucl/apps/pkg-store/castep/castep-23.1.tar.gz pretty early in this lot.

==> [2024-09-18-12:41:08.277618] Checking existence of file:///shared/ucl/apps/pkg-store/castep/castep-23.1.tar.gz
==> [2024-09-18-12:41:08.286005] Fetching file:///shared/ucl/apps/pkg-store/castep/castep-23.1.tar.gz
==> [2024-09-18-12:41:08.400952] Fetching from https://binaries.spack.io/develop/developer-tools-manylinux2014/_source-cache/archive/7f/7fba0450d3fd71586c8498ce51975bbdde923759ab298a656409280c29bf45b5.tar.gz failed.
==> [2024-09-18-12:41:08.401084] Fetching from https://binaries.spack.io/develop/developer-tools-manylinux2014/castep/castep-23.1.tar.gz failed.
==> [2024-09-18-12:41:08.401126] Fetching from file:///shared/ucl/apps/spack/0.22/buildcache/_source-cache/archive/7f/7fba0450d3fd71586c8498ce51975bbdde923759ab298a656409280c29bf45b5.tar.gz failed.
==> [2024-09-18-12:41:08.401221] Fetching from file:///shared/ucl/apps/spack/0.22/buildcache/castep/castep-23.1.tar.gz failed.
==> [2024-09-18-12:41:08.401267] Fetching from file:///shared/ucl/apps/pkg-store/_source-cache/archive/7f/7fba0450d3fd71586c8498ce51975bbdde923759ab298a656409280c29bf45b5.tar.gz failed.
==> [2024-09-18-12:41:08.402594] Checking existence of https://binaries.spack.io/develop/developer-tools-manylinux2014/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:08.745006] Failure reading URL: HEAD https://binaries.spack.io/develop/developer-tools-manylinux2014/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz returned 404: Not Found
==> [2024-09-18-12:41:08.745581] URL does not exist: https://binaries.spack.io/develop/developer-tools-manylinux2014/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:08.746209] FailedDownloadError: Failed to fetch file from URL: https://binaries.spack.io/develop/developer-tools-manylinux2014/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:08.746724] Checking existence of https://binaries.spack.io/develop/developer-tools-manylinux2014/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz 
==> [2024-09-18-12:41:09.082406] Failure reading URL: HEAD https://binaries.spack.io/develop/developer-tools-manylinux2014/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz returned 404: Not Found
==> [2024-09-18-12:41:09.082835] URL does not exist: https://binaries.spack.io/develop/developer-tools-manylinux2014/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz   
==> [2024-09-18-12:41:09.083382] FailedDownloadError: Failed to fetch file from URL: https://binaries.spack.io/develop/developer-tools-manylinux2014/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:09.083786] Checking existence of file:///shared/ucl/apps/spack/0.22/buildcache/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.084537] Failure reading URL: <urlopen error [Errno 2] No such file or directory: '/shared/ucl/apps/spack/0.22/buildcache/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz'>
==> [2024-09-18-12:41:09.084871] URL does not exist: file:///shared/ucl/apps/spack/0.22/buildcache/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.085435] FailedDownloadError: Failed to fetch file from URL: file:///shared/ucl/apps/spack/0.22/buildcache/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.085969] Checking existence of file:///shared/ucl/apps/spack/0.22/buildcache/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:09.086559] Failure reading URL: <urlopen error [Errno 2] No such file or directory: '/shared/ucl/apps/spack/0.22/buildcache/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz'>
==> [2024-09-18-12:41:09.086789] URL does not exist: file:///shared/ucl/apps/spack/0.22/buildcache/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:09.087219] FailedDownloadError: Failed to fetch file from URL: file:///shared/ucl/apps/spack/0.22/buildcache/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:09.087649] Checking existence of file:///shared/ucl/apps/pkg-store/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.088269] Failure reading URL: <urlopen error [Errno 2] No such file or directory: '/shared/ucl/apps/pkg-store/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz'>
==> [2024-09-18-12:41:09.088559] URL does not exist: file:///shared/ucl/apps/pkg-store/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.088269] Failure reading URL: <urlopen error [Errno 2] No such file or directory: '/shared/ucl/apps/pkg-store/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz'>
==> [2024-09-18-12:41:09.088559] URL does not exist: file:///shared/ucl/apps/pkg-store/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.089049] FailedDownloadError: Failed to fetch file from URL: file:///shared/ucl/apps/pkg-store/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.089538] Checking existence of file:///shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:09.090064] Failure reading URL: <urlopen error [Errno 2] No such file or directory: '/shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz'>
==> [2024-09-18-12:41:09.090259] URL does not exist: file:///shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:09.090656] FailedDownloadError: Failed to fetch file from URL: file:///shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz 
==> [2024-09-18-12:41:09.091030] Checking existence of https://mirror.spack.io/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.631688] Failure reading URL: HEAD https://mirror.spack.io/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz returned 404: Not Found
==> [2024-09-18-12:41:09.632124] URL does not exist: https://mirror.spack.io/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.632677] FailedDownloadError: Failed to fetch file from URL: https://mirror.spack.io/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz
==> [2024-09-18-12:41:09.633083] Checking existence of https://mirror.spack.io/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:10.180514] Failure reading URL: HEAD https://mirror.spack.io/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz returned 404: Not Found
==> [2024-09-18-12:41:10.180880] URL does not exist: https://mirror.spack.io/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:10.181360] FailedDownloadError: Failed to fetch file from URL: https://mirror.spack.io/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> [2024-09-18-12:41:10.181710] Checking existence of file:///lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/Castep_23.1_build_fixes.diff.gz
==> [2024-09-18-12:41:10.182276] Failure reading URL: <urlopen error [Errno 2] No such file or directory: '/lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/Castep_23.1_build_fixes.diff.gz'>
==> [2024-09-18-12:41:10.182511] URL does not exist: file:///lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/Castep_23.1_build_fixes.diff.gz
==> [2024-09-18-12:41:10.182929] FailedDownloadError: Failed to fetch file from URL: file:///lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/Castep_23.1_build_fixes.diff.gz
==> [2024-09-18-12:41:10.183161] Fetching from https://binaries.spack.io/develop/developer-tools-manylinux2014/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz failed.
==> [2024-09-18-12:41:10.183428] Fetching from https://binaries.spack.io/develop/developer-tools-manylinux2014/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz failed. 
==> [2024-09-18-12:41:10.183597] Fetching from file:///shared/ucl/apps/spack/0.22/buildcache/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz failed.
==> [2024-09-18-12:41:10.183918] Fetching from file:///shared/ucl/apps/spack/0.22/buildcache/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz failed.
==> [2024-09-18-12:41:10.184125] Fetching from file:///shared/ucl/apps/pkg-store/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz failed.
==> [2024-09-18-12:41:10.184395] Fetching from file:///shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz failed.
==> [2024-09-18-12:41:10.184561] Fetching from https://mirror.spack.io/_source-cache/archive/5f/5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2.gz failed.
==> [2024-09-18-12:41:10.184734] Fetching from https://mirror.spack.io/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz failed.
==> [2024-09-18-12:41:10.184840] Fetching from file:///lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/Castep_23.1_build_fixes.diff.gz failed.
==> [2024-09-18-12:41:10.199753] Error: FetchError: Manual download is required for castep. Refer to http://castep.org for download instructions.
Traceback (most recent call last):
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/build_environment.py", line 1151, in _setup_pkg_and_run
    return_value = function(pkg, kwargs)
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/installer.py", line 2482, in build_process
    return installer.run()
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/installer.py", line 2331, in run
    self.pkg.do_patch()
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/package_base.py", line 1533, in do_patch
    self.do_stage()
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/package_base.py", line 1521, in do_stage
    self.do_fetch(mirror_only)
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/package_base.py", line 1505, in do_fetch
    self.stage.fetch(mirror_only, err_msg=err_msg)
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/util/pattern.py", line 16, in __call__
    return [getattr(item, self.name)(*args, **kwargs) for item in self.container]
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/util/pattern.py", line 16, in <listcomp>
    return [getattr(item, self.name)(*args, **kwargs) for item in self.container]
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/stage.py", line 562, in fetch
    raise spack.error.FetchError(err_msg or default_msg, None)
spack.error.FetchError: Manual download is required for castep. Refer to http://castep.org for download instructions.
==> [2024-09-18-12:41:10.200988] Flagging castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2 as failed: FetchError: Manual download is required for castep. Refer to http://castep.org for download instructions.
==> [2024-09-18-12:41:10.225223] Downgrading to a read lock on castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2 with timeout 1.000ns
==> [2024-09-18-12:41:10.225422] castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2 is now read locked
...
==> [2024-09-18-12:41:10.273850] Releasing read lock on castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2
==> [2024-09-18-12:41:10.274679] Error: castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2: FetchError: Manual download is required for castep. Refer to http://castep.org for download instructions.
==> [2024-09-18-12:41:10.274708] Error: castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2: Package was not installed
==> [2024-09-18-12:41:10.274728] Associating installation failure with first failed explicit package (castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2) from castep-23.1-s4ypamfv2gzuc34al3i6kjxz73l3zll2
==> [2024-09-18-12:41:10.274833] InstallError: Installation request failed.  Refer to reported errors for failing package(s).
==> [2024-09-18-12:41:10.274852] Error: Installation request failed.  Refer to reported errors for failing package(s).
Traceback (most recent call last):
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/main.py", line 1069, in main
    return _main(argv)
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/main.py", line 1022, in _main
    return finish_parse_and_run(parser, cmd_name, args, env_format_error)
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/main.py", line 1052, in finish_parse_and_run
    return _invoke_command(command, parser, args, unknown) 
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/main.py", line 649, in _invoke_command
    return_val = command(parser, args)
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/cmd/install.py", line 352, in install
    install_with_active_env(env, args, install_kwargs, reporter_factory)
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/cmd/install.py", line 421, in install_with_active_env
    env.install_specs(specs_to_install, **install_kwargs)  
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/environment/environment.py", line 1957, in install_specs
    PackageInstaller(installs).install()
  File "/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/lib/spack/spack/installer.py", line 2252, in install
    raise InstallError(
spack.installer.InstallError: Installation request failed.  Refer to reported errors for failing package(s).
heatherkellyucl commented 1 month ago

It appears to be /shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz which is the problem because it is only trying to get file:///shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz ?

heatherkellyucl commented 1 month ago

Ok, it makes /shared/ucl/apps/spack/0.22/hk-initial-stack/build_stage/ccspapp/spack-stage-patch-5f31daf4733f8ee906cba0ff092e317b9cbaa100666533b2dce39f1f829646c2 at the same time and does not put anything into it.

I put a renamed copy of the patch file with the short hash into /shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz where it was looking and then it can proceed.

Now I need to make 23.1 its own castep_cmake_libxc.patch because it is libxc 5.2.2 there and not 5.2.3 so the directory is different.

heatherkellyucl commented 1 month ago

23.1 is also successfully built and has both Castep's patch and my patch applied.

==> Installing castep-23.1-5ed5smvptchqxptpm737r6cfhzuxeyvo [103/104]
==> No binary for castep-23.1-5ed5smvptchqxptpm737r6cfhzuxeyvo found: installing from source
==> Fetching file:///shared/ucl/apps/pkg-store/castep/castep-23.1.tar.gz
==> Fetching file:///shared/ucl/apps/pkg-store/castep/Castep_23.1_build_fixes.diff.gz-5f31daf.gz
==> Applied patch file:///lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/Castep_23.1_build_fixes.diff.gz
==> Applied patch /lustre/scratch/scratch/ccspapp/spack/0.22/hpc-spack/repos/dev/packages/castep/castep_cmake_libxc522.patch
==> castep: Executing phase: 'cmake'
==> castep: Executing phase: 'build'
==> castep: Executing phase: 'install'
==> castep: Successfully installed castep-23.1-5ed5smvptchqxptpm737r6cfhzuxeyvo
  Stage: 18.41s.  Cmake: 1m 48.79s.  Build: 9m 23.49s.  Install: 5.51s.  Post-install: 1.39s.  Total: 11m 43.42s
heatherkellyucl commented 1 month ago

Testing 24.1 including mpi:

cd /home/ccspapp/Scratch/castep/CASTEP-24.1/Test
make CASTEPEXE=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-24.1-c2u26cbdn7nrrq4qt3me4acvtr2cdfrp/bin/castep.mpi COMMS_ARCH=mpi
Found mpirun, using PARALLEL_CMD=mpirun -np tc.nprocs
rm -f */*/*.castep */*/*.dfpt_wvfn */*/*.fd_wvfn */*/*.wvfn.* */*/*.*.err
../bin/testcode.py -q  --processors=4 --total-processors=36 --user-option castep run_cmd_template "../../../bin/run_castep_test.pl tc.program tc.args tc.input tc.output mpirun -np tc.nprocs" -e /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-24.1-c2u26cbdn7nrrq4qt3me4acvtr2cdfrp/bin/castep.mpi -c simple -c d3-simple -c d4-simple -c solvation-simple -c md-parallel
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ [496/496]

Last night I accidentally ran the castep 23.1 binary with 24.1's tests and got 20 failures (476/496 passed). Which is actually slightly reassuring.

Then the libxc tests:

make check-libxc CASTEPEXE=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-24.1-c2u26cbdn7nrrq4qt3me4acvtr2cdfrp/bin/castep.mpi COMMS_ARCH=mpi
Found mpirun, using PARALLEL_CMD=mpirun -np tc.nprocs
rm -f */*/*.castep */*/*.dfpt_wvfn */*/*.fd_wvfn */*/*.wvfn.* */*/*.*.err
../bin/testcode.py -q  --processors=4 --total-processors=36 --user-option castep run_cmd_template "../../../bin/run_castep_test.pl tc.program tc.args tc.input tc.output mpirun -np tc.nprocs" -e /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-24.1-c2u26cbdn7nrrq4qt3me4acvtr2cdfrp/bin/castep.mpi -c libxc-simple
.... [4/4]
heatherkellyucl commented 1 month ago

23.1 also passes:

cd /home/ccspapp/Scratch/castep/CASTEP-23.1/Test
make CASTEPEXE=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-23.1-5ed5smvptchqxptpm737r6cfhzuxeyvo/bin/castep.mpi COMMS_ARCH=mpi
Found mpirun, using PARALLEL_CMD=mpirun -np tc.nprocs
rm -f */*/*.castep */*/*.dfpt_wvfn */*/*.fd_wvfn */*/*.wvfn.* */*/*.*.err
../bin/testcode.py -q  --processors=4 --total-processors=36 --user-option castep run_cmd_template "../../../bin/run_castep_test.pl tc.program tc.args tc.input tc.output mpirun -np tc.nprocs" -e /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-23.1-5ed5smvptchqxptpm737r6cfhzuxeyvo/bin/castep.mpi -c simple -c d3-simple -c d4-simple -c md-parallel
....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... [487/487]
make check-libxc CASTEPEXE=/lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-23.1-5ed5smvptchqxptpm737r6cfhzuxeyvo/bin/castep.mpi COMMS_ARCH=mpi
Found mpirun, using PARALLEL_CMD=mpirun -np tc.nprocs
rm -f */*/*.castep */*/*.dfpt_wvfn */*/*.fd_wvfn */*/*.wvfn.* */*/*.*.err
../bin/testcode.py -q  --processors=4 --total-processors=36 --user-option castep run_cmd_template "../../../bin/run_castep_test.pl tc.program tc.args tc.input tc.output mpirun -np tc.nprocs" -e /lustre/shared/ucl/apps/spack/0.22/hk-initial-stack/spack/opt/spack/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/__spack_path_placeholder__/linux-rhel7-cascadelake/gcc-12.3.0/castep-23.1-5ed5smvptchqxptpm737r6cfhzuxeyvo/bin/castep.mpi -c libXC-simple
.. [2/2]
heatherkellyucl commented 4 weeks ago

I have added castep 23 and 24 +libxc into base.yaml.

Things that still need doing in the recipe: