Question: How to check local cache before Spack mirror?

[christoph-conrads]

Spack is running on a cluster without direct internet access and all attempts to connect to the spack binary mirror at https://mirror.spack.io fail of course after some time before Spack checks its source package cache. Is it possible to disable this mirror or to make Spack check its own cache first?

The documentation shows in the section on spack mirror remove the following example:

$ spack mirror remove local_filesystem
$ spack mirror list
==> No mirrors configured.

This does not work for me:

$ spack mirror list
spack-public    https://mirror.spack.io
$ spack mirror remove spack-public
==> Error: No mirror with name spack-public
$ spack debug report
* **Spack:** 0.17.1
* **Python:** 3.6.8
* **Platform:** linux-rhel8-aarch64
* **Concretizer:** clingo

edit

An example with 1:20 minutes of waiting time:

==> [2022-01-14-17:32:17.235378] Installing cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn
==> [2022-01-14-17:32:17.235568] Searching for binary cache of cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn
==> [2022-01-14-17:33:37.390660] Did not find linux-rhel8-aarch64-gcc-11.1.0-cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn.spec.json on https://mirror.spack.io/build_cache/linux-rhel8-aarch64-gcc-11.1.0-cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn.spec.json
  Download failed: <urlopen error timed out>
==> [2022-01-14-17:33:37.391003] Did not find linux-rhel8-aarch64-gcc-11.1.0-cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn.spec.json on https://mirror.spack.io/build_cache/linux-rhel8-aarch64-gcc-11.1.0-cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn.spec.yaml
  Download failed: <urlopen error timed out>
==> [2022-01-14-17:33:37.391285] No binary for cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn found: installing from source
==> [2022-01-14-17:33:37.408880] Creating the installation directory /ccc/work/cont003/gen13127/conradsc/spack/opt/spack/linux-rhel8-aarch64/gcc-11.1.0/cgns-4.2.0-glconkz5jyxdy2qse5hxopo65ny6gssn

[hainest]

I'm having this same problem on Ubuntu 18.04 using spack@devlop 3540f8200a5. ~Additionally, the spack-public mirror is using the wrong OS.~ I see this part was asked and answered yesterday (https://github.com/spack/spack/issues/28407).

$ spack mirror list
spack-public    https://mirror.spack.io

$ cat ~/.spack/mirrors.yaml
mirrors: {}

$ spack mirror add E4S https://cache.e4s.io
E4S             https://cache.e4s.io (fetch)
E4S             https://cache.e4s.io (push)
spack-public    https://mirror.spack.io

# Make sure everything is clean
$ spack clean
==> Removing all temporary build stages

$ spack clean -b
==> Removing bootstrapped software and configuration in "MYHOME/.spack/bootstrap"

$ spack install --cache-only libz
==> Bootstrapping clingo from pre-built binaries
==> Fetching https://mirror.spack.io/bootstrap/github-actions/v0.1/build_cache/linux-rhel5-x86_64/gcc-9.3.0/clingo-bootstrap-spack/linux-rhel5-x86_64-gcc-9.3.0-clingo-bootstrap-spack-vcipwnf57slgoo7busvvkzjkk7vydeb5.spack
==> Installing "clingo-bootstrap@spack%gcc@9.3.0~docs~ipo+python build_type=Release arch=linux-rhel5-x86_64" from a buildcache
==> Bootstrapping patchelf from pre-built binaries
==> Installing patchelf-0.13.1-2kojfp6zrwkjsjvxo6qz76zfsppve2v2
==> No binary for patchelf-0.13.1-2kojfp6zrwkjsjvxo6qz76zfsppve2v2 found: installing from source
==> Error: Failed to install patchelf due to PermissionError: [Errno 13] Permission denied: 'MYHOME.spack/bootstrap/store/linux-ubuntu18.04-x86_64/gcc-8.4.0/patchelf-0.13.1-2kojfp6zrwkjsjvxo6qz76zfsppve2v2'
==> Bootstrapping clingo from sources
^C

It's trying to bootstrap linux-rhel5, but my system is linux-ubuntu18.04 as correctly detected when installing patchelf. This fails because there is no bootstrap store for ubuntu.

[alalazo]

@hainest Binaries for linux are installed from rhel5 because they have been built on a (slightly modified) manylinux1 image.

[christoph-conrads]

Work-around

Take the code from PR #28556, add spack/var/spack/cache/metadata/sources/metadata.yaml with the contents below, and add the cache as a mirror: spack bootstrap add --trust local-sources /ABSOLUTE/path/to/spack/var/spack/cache/metadata/sources/metadata.yaml.

metadata/sources/metadata.yaml:

type: install
description: Mirror with software needed to bootstrap Spack
info:
  url: file:///ignored

[christoph-conrads]

Work-around:

diff --git a/lib/spack/spack/installer.py b/lib/spack/spack/installer.py
index df1d704..61f7a9c 100644
--- a/lib/spack/spack/installer.py
+++ b/lib/spack/spack/installer.py
@@ -275,8 +275,7 @@ def _install_from_cache(pkg, cache_only, explicit, unsigned=False,
         bool: ``True`` if the package was extract from binary cache,
             ``False`` otherwise
     """
-    installed_from_cache = _try_install_from_binary_cache(
-        pkg, explicit, unsigned=unsigned, full_hash_match=full_hash_match)
+    installed_from_cache = False
     pkg_id = package_id(pkg)
     if not installed_from_cache:
         pre = 'No binary for {0} found'.format(pkg_id)