add tests from spack test for E4S/22.11

[shahzebsiddiqui]

list of available spack environments

```console
siddiq90@login01> spack env list
==> 4 environments
    cce  cuda  gcc  nvhpc

Here is list of available tests

available tests in cuda environments

siddiq90@login01> spack test list
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
cmh6tle amrex@22.11          o7d6crd cmake@3.24.2  c3nzkgk hypre@2.26.0   jshtkzb libxml2@2.10.1   kp4vgrk perl@5.36.0    u3qpmib slate@2022.07.00  egmduil strumpack@7.0.1     mhyaalu superlu-dist@8.1.1  ymsz6uq vtk-m@1.9.0
t6sxq2b arborx@1.3           hsje3i5 ginkgo@1.4.0  vg6ezbk hypre@2.26.0   rlvc53b m4@1.4.18        z7ox3vz petsc@3.18.1   ain6rsg slepc@3.18.1      w5dffgj sundials@6.4.1      ibk4pur sz@2.1.12.2
gqdvawb berkeley-db@18.1.40  jihwymm hdf5@1.12.2   ryw5vwf kokkos@3.7.00  rfs4zxy mfem@4.5.0       lcmhz65 petsc@3.18.1   veeosbb sqlite@3.39.4     okaupsq superlu-dist@7.2.0  bvytqoh tasmanian@7.9
nkjsd63 caliper@2.8.0        paueqpz heffte@2.3.0  apv6gpx kokkos@3.7.00  kal7ug2 omega-h@9.34.13  5sqhnro python@3.9.15  mbi2n6r strumpack@7.0.1   j44lhov superlu-dist@8.1.1  ya4665r umpire@2022.03.1

available tests in gcc environment

siddiq90@login01> spack test list
-- linux-sles15-zen3 / gcc@11.2.0 -------------------------------
tvevpst aml@0.2.0              zv5bs3l legion@21.03.0          54seods py-cffi@1.15.0                mfrljom py-h5py@3.7.0               sjasus7 py-numpy@1.21.5           h5qyj3t py-pybind11@2.8.1          jdnhcgh py-scipy@1.8.1           kfbhday slepc@3.18.1
yotm6qv amrex@22.11            ni76haj libsigsegv@2.13         fj2jfzz py-cffi@1.15.0                f4p576n py-hpbandster@0.7.4         4ybolda py-numpy@1.23.4           emjnaqa py-pybind11@2.8.1          nn4o6hx py-serpent@1.40          veeosbb sqlite@3.39.4
fcakmeo arborx@1.3             jshtkzb libxml2@2.10.1          vjza3kn py-charset-normalizer@2.0.12  ggljeu5 py-idna@3.4                 g2fka4j py-opentuner@0.8.7        vjhfb62 py-pycparser@2.21          bmabmuo py-six@1.16.0            mbi2n6r strumpack@7.0.1
gqdvawb berkeley-db@18.1.40    rlvc53b m4@1.4.18               xye7so6 py-cloudpickle@2.2.0          xuvrrwj py-ipykernel@6.16.0         5cwycdg py-packaging@21.3         tsol64i py-pycparser@2.21          a5fgql3 py-sqlalchemy@1.4.20     hpe3vx7 sundials@6.4.1
eylvvre binutils@2.38          nro7oe4 mfem@4.5.0              pbe6q5w py-configspace@0.4.20         idhyxm4 py-ipyparallel@6.3.0        x6eauvn py-pandas@1.5.1           aqob7gl py-pyelftools@0.26         vwpi6ih py-stack-data@0.5.0      tnurd2u superlu@5.3.0
346urvd bolt@2.0               smslqc6 omega-h@9.34.13         7sxr6td py-contourpy@1.0.5            x7nsewi py-ipython@8.5.0            c2fa4qf py-paramz@0.9.5           7agstov py-pygments@2.13.0         o66sobq py-statsmodels@0.13.2    okaupsq superlu-dist@7.2.0
grndums caliper@2.8.0          27fkqq2 openpmd-api@0.14.5      ti33v42 py-cycler@0.11.0              luk4rhr py-ipython-genutils@0.2.0   of2btgm py-parso@0.8.3            gfe4jqn py-pymoo@0.5.0             wwey2d3 py-threadpoolctl@3.1.0   j44lhov superlu-dist@8.1.1
o7d6crd cmake@3.24.2           r2mgpa3 openpmd-api@0.14.5      n7m2hr2 py-cython@0.29.32             n3mkaxg py-jedi@0.18.1              ulhzdto py-patsy@0.5.2            v62tm6o py-pyparsing@3.0.9         xkv66ph py-toml@0.10.2           rhuu637 swig@4.0.2
6v4ndhw fortrilinos@2.1.0      s7jp42s papyrus@1.0.2           3jn3luz py-debugpy@1.6.3              yqck2vt py-joblib@1.2.0             ofngmsq py-periodictable@1.5.0    xy4posc py-pyparsing@3.0.9         kcl3y2e py-tornado@6.2           ibk4pur sz@2.1.12.2
4qbfhkq gasnet@2022.9.0        mlpz3o2 parallel-netcdf@1.12.3  o5pkbeo py-decorator@5.1.1            qmr2jr6 py-jsonschema@4.16.0        t4mdaej py-petsc4py@3.18.1        svpzglj py-pyro4@4.81              hjettne py-traitlets@5.3.0       4aoihbr tasmanian@7.9
zsf4hui ginkgo@1.4.0           fkn7h2o parsec@3.0.2209         tczb3l6 py-entrypoints@0.4            4zqnab7 py-jupyter-client@7.3.5     mt74d4d py-pexpect@4.8.0          crtvyvq py-pyrsistent@0.18.1       5zgacnv py-urllib3@1.26.12       fqug27d tau@2.32
45363uk gptune@3.0.0           kp4vgrk perl@5.36.0             f22qzae py-executing@1.1.0            4syeydn py-jupyter-core@4.11.1      hcwi2ez py-pickleshare@0.7.5      zshj37p py-python-dateutil@2.8.2   xyblxob py-warpx@22.10           ij2fhlm umpire@2022.03.1
xq5v2c3 hdf5@1.8.21            k3ngav3 petsc@3.18.1            cpv2ltw py-filelock@3.5.0             knet3fd py-kiwisolver@1.3.2         hyamcu3 py-picmistandard@0.0.20   nmbbxhp py-pythran@0.11.0          xjmhipk py-warpx@22.10           vlrnu6v umpire@2022.03.1
jihwymm hdf5@1.12.2            zn7tuho pumi@2.2.7              cobytwi py-fn-py@0.5.2                kvssohu py-lhsmdu@1.1               xqgaci2 py-pillow@9.2.0           6yblafv py-pythran@0.11.0          ypvqjjb py-warpx@22.10           cge4wqs upcxx@2022.9.0
5svtlvy hdf5@1.12.2            zykjeu4 py-asttokens@2.0.8      66uakew py-fonttools@4.37.3           qq3aj74 py-libensemble@0.9.3        64uc5ec py-ply@3.11               bkaspyt py-pytz@2022.2.1           jucjstw py-wcwidth@0.2.5         qab3xjp vtk-m@1.9.0
me7wpex hdf5@1.12.2            giyra7h py-attrs@22.1.0         4guv5rj py-future@0.18.2              cwncpzp py-matplotlib@3.6.2         joh3id4 py-ply@3.11               2iauo3q py-pyyaml@6.0              hi3cc5f py-ytopt-autotune@1.1.0  wfetvq7 warpx@22.10
6cqopmp hdf5@1.13.2            3vcptdf py-autograd@1.3         nkxtec2 py-gast@0.5.3                 wywh3f6 py-matplotlib-inline@0.1.3  fhferxy py-prompt-toolkit@3.0.31  563caon py-pyyaml@6.0              7gnip4c python@3.8.15            kvj4ka6 warpx@22.10
75xngjp heffte@2.3.0           5gyseug py-backcall@0.2.0       vblsxon py-gast@0.5.3                 fgq7z4m py-mpi4py@3.1.3             7ay2has py-psutil@5.9.2           ei6a2w3 py-pyzmq@24.0.1            5sqhnro python@3.9.15            6bezcml warpx@22.10
uw6cz6o hpctoolkit@2022.10.01  sy3wqk6 py-beniget@0.4.1        w4axal6 py-gevent@1.5.0               234rptv py-mpi4py@3.1.3             wpsn6fi py-psutil@5.9.2           2fd7c6t py-requests@2.28.1         wl7p5wm python@3.9.15
c3nzkgk hypre@2.26.0           a65puq5 py-beniget@0.4.1        fkxwdff py-gpy@1.9.9                  eqazxcg py-nest-asyncio@1.5.5       mz6eyuf py-ptyprocess@0.7.0       n5pm26l py-scikit-learn@1.1.3      5ucvryz qthreads@1.16
kapfuog kokkos@3.7.00          sb4v63m py-bottleneck@1.3.5     vdan4xn py-greenlet@1.1.3             4ijlhpl py-netifaces@0.10.5         fsehbnw py-pure-eval@0.2.2        k3d3qkd py-scikit-optimize@master  yae4jh3 raja@2022.03.0
qe4wcis kokkos@3.7.00          d5amyu6 py-certifi@2022.9.14    mllwxuw py-h5py@3.7.0                 aojwxn3 py-numexpr@2.8.3            ffyoo7c py-pyaml@21.8.3           6zhjt2n py-scipy@1.8.1             d3zj4mo slate@2022.07.00

[shahzebsiddiqui]

@wspear can you help with sending a template or list of packages we need to test for this stack

[wspear]

@shahzebsiddiqui I should be able to get a PR together for this tomorrow. I've created a very basic (possibly not very robust) script that takes the intersection of spack find -x andspack test list. This is what it turns out for e4s/22.11%gcc, which I'll use to generate the new buildtests unless you have input to the contrary.

aml
amrex
arborx
bolt
caliper
fortrilinos
gasnet
ginkgo
gptune
heffte
hpctoolkit
kokkos
legion
mfem
omega-h
openpmd-api
papyrus
parallel-netcdf
parsec
pumi
py-h5py
py-libensemble
py-petsc4py
py-warpx
qthreads
slate
slepc
sundials
superlu
tasmanian
tau
upcxx
vtk-m

[shahzebsiddiqui]

i also posted an issue to spack for feature spack test list -x and spack test list -X to help address this use-case https://github.com/spack/spack/issues/37025

[wspear]

This is the template I'm considering for these tests:

#This buildtest was generated from spack-test-22.11-template.yaml
buildspecs:
  {package}_spack-test_22.11:
    type: spack
    executor: perlmutter.slurm.regular
    description: Run spack test for {package} from e4s/22.11 gcc stack
    tags: [e4s]
    sbatch: ["-t 30", "-N 1", "-G 1", "-A m3503_g", "-C gpu"]
    tags: e4s
    pre_cmds: |
      module load e4s/22.11
      spack env activate gcc
      spack load --first cmake
      spack load --first {package}
    spack:
      root: /global/common/software/spackecp/perlmutter/e4s-22.11/default/spack
      verify_spack: false
      test:
        run:
          specs: ['{package}']
        results:
          option: '-l'
          specs: ['{package}']
maintainers:
  - shahzebsiddiqui
  - wspear

An issue to consider: there are some duplicate packages in these e4s deployments. When more than one package of the same name is installed I'm not sure how it will behave with only the 'first' found version loaded. Alternatives are to not load the package at all and consider requiring pre-loading the package to be an issue with the test or to do a more advanced extraction of installed specs and create separate buildspecs that specify the hash for each test to be run.

[wspear]

PR is here. Thanks to the template, making any changes across all the new tests will be quick. https://software.nersc.gov/NERSC/buildtest-nersc/-/merge_requests/164