gmao-cda
commented
8 months ago Another installation error related to crtm@2.4.0.1 (Solved)
I believe this error of crtm is unrelated to my machine, but related to to-be-built package itself. Error info is:
[+] /home/cda/fast/pkg/spack-stack/envs/unified-env.jojoSOCA/install/gcc/11.4.0/ncio-1.1.2-pvmcajg
==> Installing crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok [265/399]
==> No binary for crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok found: installing from source
==> Warning: Fetching from mirror without a checksum!
This package is normally checked out from a version control system, but it has been archived on a spack mirror. This means we cannot know a checksum for the tarball in advance. Be sure that your connection to this mirror is secure!
gzip: stdin: invalid compressed data--crc error
gzip: stdin: invalid compressed data--length error
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Unexpected EOF in archive
/usr/bin/tar: Error is not recoverable: exiting now
==> Using cached archive: /home/cda/fast/pkg/spack-stack/cache/source_cache/_source-cache/git//JCSDA/crtm.git/7ecad4866c400d7d0db1413348ee225cfa99ff36.tar.gz
==> Error: ProcessError: Command exited with status 2:
'/usr/bin/tar' '-oxf' '/home/cda/fast/pkg/spack-stack/cache/build_stage/spack-stage-crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok/7ecad4866c400d7d0db1413348ee225cfa99ff36.tar.gz'
==> Warning: Skipping build of gsi-env-1.0.0-372rrkrny3wsxhvgrup7u3k7zd43oghj since crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok failed
==> Warning: Skipping build of upp-10.0.10-2sj4tx2v3cpocqqrtkgfz3taoave6yjh since crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok failed
==> Warning: Skipping build of ufs-srw-app-env-1.0.0-5hvlqzo4c5ckntcupnk5gl33m5zx5w7h since upp-10.0.10-2sj4tx2v3cpocqqrtkgfz3taoave6yjh failed
==> Warning: Skipping build of global-workflow-env-1.0.0-babe3raht6rts7hpd235rkak5cvnjbkt since crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok failed
==> Warning: Skipping build of ufs-weather-model-env-1.0.0-xyjxfabdvbiz3cj4vdcbzxxxzjghmsal since crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok failed
==> Error: Terminating after first install failure: ProcessError: Command exited with status 2:
'/usr/bin/tar' '-oxf' '/home/cda/fast/pkg/spack-stack/cache/build_stage/spack-stage-crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok/7ecad4866c400d7d0db1413348ee225cfa99ff36.tar.gz'The key error message (last line) shows tar fails for the file /home/cda/fast/pkg/spack-stack/cache/build_stage/spack-stage-crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok/7ecad4866c400d7d0db1413348ee225cfa99ff36.tar.gz
Debug info
I tried to manually run the last tar command, with -v option, so I know what happens
cda@jojo:~/Desktop$ tar -xvzf 7ecad4866c400d7d0db1413348ee225cfa99ff36.tar.gz
crtm/
crtm/.gitattributes
crtm/.gitignore
crtm/CMakeLists.txt
crtm/COPYING
crtm/LICENSE.md
crtm/NOTES.md
crtm/README.md
crtm/VERSION
crtm/crtm_release_notes.txt
crtm/.github/
crtm/.github/main.yaml
crtm/cmake/
crtm/cmake/CTestCustom.ctest.in
crtm/cmake/FindNetCDF.cmake
crtm/cmake/PackageConfig.cmake.in
crtm/cmake/compiler_flags_GNU_Fortran.cmake
crtm/cmake/compiler_flags_Intel_Fortran.cmake
crtm/cmake/crtm_compiler_flags.cmake
crtm/fix/
(...Cheng Da's comment: skip showing all here...)
crtm/fix/TauCoeff/ODPS/Little_Endian/iasi616_metop-c.TauCoeff.bin
crtm/fix/TauCoeff/ODPS/Little_Endian/iasiB1_metop-a.TauCoeff.bin
gzip: stdin: invalid compressed data--crc error
gzip: stdin: invalid compressed data--length error
tar: Unexpected EOF in archive
tar: Unexpected EOF in archive
tar: Error is not recoverable: exiting now
Seems it's the package /home/cda/fast/pkg/spack-stack/cache/build_stage/spack-stage-crtm-2.4.0.1-fg456hia2nbwycn6wkgq6kjvk7hf7sok/7ecad4866c400d7d0db1413348ee225cfa99ff36.tar.gz has the problem.
Traceback & Possible reasons for this failure
Required by following packages
repos/spack-stack/packages/ufs-weather-model-env/package.py: depends_on("crtm@2.4.0.1", type="run")
repos/spack-stack/packages/ufs-srw-app-env/package.py: depends_on("crtm@2.4.0.1")
repos/spack-stack/packages/global-workflow-env/package.py: depends_on("crtm@2.4.0.1")
repos/spack-stack/packages/jedi-base-env/package.py: # depends_on("crtm@v2.4.1-jedi", type="run")
repos/spack-stack/packages/gmao-swell-env/package.py: depends_on("crtm@v2.4-jedi.2", type="run")
repos/spack-stack/packages/gsi-env/package.py: depends_on("crtm@2.4.0.1")Checking repos/builtin/packages/crtm/package.py shows that
crtm@2.4.0.1 corresponds to version("2.4.0.1", tag="v2.4.0_emc.3", commit="7ecad4866c400d7d0db1413348ee225cfa99ff36")
Let me see what happens if I directly download from github
Update 1:
- 02.24: I cannot download it completely from the github page (https://github.com/JCSDA/crtm/tree/v2.4.0_emc.3). Maybe some issues with Github today. Will retry tomorrow
- 02.25: I have tried again, still the same problem
- 02.26: Failed again using UMD network. I notice this tar.gz package is several GBs (because it includes CRTM coeffs). is it possible that this repo has used up the bandwidth this month, since this large package will be regarded a lfs? So that I cannot download it now
- 03.18: crtm can finally be downloaded completely from github now & finished installing the spack-stack without neglecting apps related to crtm in
spack.yaml
Update 2: temporary fix by neglecting apps related to crtm@2.4.0.1:
created a customerized template soca-dev, with its spack.yaml as
# spack-stack hash: 06de425
# spack hash: 03ecae6b77
spack:
concretizer:
unify: when_possible
view: false
include:
- site
- common
definitions:
- compilers: ['%gcc']
- packages:
#- ewok-env +ecflow +cylc
#- geos-gcm-env
#- global-workflow-env
#- gmao-swell-env
#- gsi-env
#- jedi-fv3-env
#- jedi-geos-env
#- jedi-mpas-env
#- jedi-neptune-env
- jedi-tools-env
#- jedi-ufs-env
#- jedi-um-env
#- nceplibs-env
- soca-env
#- ufs-srw-app-env
#- ufs-utils-env
#- ufs-weather-model-env
#- upp-env
#- ww3-env
# Various fms tags (list all to avoid duplicate packages)
- fms@release-jcsda
- fms@2023.04
# Various crtm tags (list all to avoid duplicate packages)
#- crtm@2.4.0.1
#- crtm@v2.4.1-jedi
# MADIS for WCOSS2 decoders.
- madis@4.5
specs:
- matrix:
- [$packages]
- [$compilers]
exclude:
# jedi-tools doesn't build with Intel
- jedi-tools-env%intel
packages:
all:
compiler: [gcc@11.4.0]
providers:
mpi: [mpich@4.1.1]
fontconfig:
variants: +pic
pixman:
variants: +pic
cairo:
variants: +picNow spack finishes installation. I hope SOCA does not use radiance so that we don't need CRTM. Let me see if this simplified works for your tutorial
travissluka
Hi Travis,
JoJo (12 cores+64GB memory) finished updating.
I'm following this tutorial (https://spack-stack.readthedocs.io/en/1.6.0/NewSiteConfigs.html#prerequisites-ubuntu-one-off) to build
spack& templateunified-devon my local (os: Kubuntu 22.04 LTS,6.5.0-21-generic #21~22.04.1-Ubuntu,gcc@11.4.0).Problem
It seems the installation prerequisites
6.2.2. Prerequisites: Ubuntu (one-off)(https://spack-stack.readthedocs.io/en/1.6.0/NewSiteConfigs.html#prerequisites-ubuntu-one-off), Step 1 "Install basic OS packages as root", Section "Misc" is not complete.I got two errors
Problem related to
autopoint:Problem related to
gettext:Solution
I Need to install the additional package
Consequence
Successfully passed these previously-failed sections, though the installation has't finished yet. I will update when I finish.
Comment
spack.yml. So many components that I don't need to use.