Is it possible to add a Spack configuration file for BOUT++?

[alephpiece]

Spack is very good at building scientific software and is useful for users who want to build BOUT++ from scratch. However, I didn't find an official configuration for BOUT++ yet. I think it could be very helpful if there is one.

I have been trying to build BOUT++ on my system with Spack for a while. It seems like almost every dependency required by BOUT++ is satisfied with packages in the Spack official repo. Here is a list:

• Autotools: satisfied with autoconf and automake
• CMake: satisfied with cmake
• MPI: satisfied with MPICH, OpenMPI, etc.
• HDF5: satisfied with hdf5
• netCDF: satisfied with netcdf-c, netcdf-cxx4
• BLAS/LAPACK: satisfied with netlib-lapack, etc. (BOUT++ is searching for libblas so the libraries provided by packages like scalapack could not be used directly)
• FFTW3: satisfied with fftw@3:
• SUNDIALS: satisfied with sundials, including CVODE, IDA, and ARKODE
• PETSc: satisfied with petsc@3.4.0:
• SLEPC: satisfied with slepc@3.4.0:
• MUMPS: satisfied with mumps (for gcc@10, the flag -fallow-argument-mismatch should be passed in)
• Score-P: satisfied with scorep
• Python packages: satisfied with py-netcdf4, etc.

I have successfully built the BOUT++ library with these packages and linked it to the example conduction.

Here is my simple configuration without a test stage.

from spack import *

class BoutDev(AutotoolsPackage):
    """BOUT++ is a framework for writing fluid and plasma simulations in curvilinear geometry.
    It is intended to be quite modular, with a variety of numerical methods and time-integration
    solvers available. BOUT++ is primarily designed and tested with reduced plasma fluid models
    in mind, but it can evolve any number of equations, with equations appearing in a readable form.
    """

    homepage = "http://boutproject.github.io/"
    url      = "https://github.com/boutproject/BOUT-dev/releases/download/v4.3.2/BOUT++-v4.3.2.tar.gz"

    # FIXME: Add a list of GitHub accounts to
    # notify when the package is updated.
    # maintainers = ['github_user1', 'github_user2']

    version('4.3.2', sha256='ee10aa9ce90fdec4d3ac0cc11bfdedadcbda19b3d0a74851f94633ddc80c5751')
    version('4.3.1', sha256='7763fb4be96dd89551a0bb3a9b435dc09ebac4ef26d80b5edaa0f7c013d1b558')
    version('4.3.0', sha256='db50a66a62edf87f04b8cb6637838811bb9307726e748a9c1979fb1cbf250cd9')
    version('4.2.3', sha256='321192c8297855be65fad8080ed880eb50338db7fff680614d430f7fef750f42')
    version('4.2.2', sha256='80518ccf8aa4d2e61fd1da96b427725ef03e481ee88d193e39eb846c5f292290')
    version('4.2.1', sha256='2e757fc15d1d52bfb80f94963d1aac95a5e910c60ec586e13ee107694a948bf6')

    variant('shared', default=False, description='Enable building bout++ into an shared object')
    variant('checks', default='none', description='Set run-time checking level',
            values=('1', '2', '3', 'none'), multi=False)
    variant('optimize', default='2', description='Enable optimization',
            values=('1', '2', '3', '4', 'none'), multi=False)
    variant('track', default=False, description='Enable variable tracking')
    variant('debug', default=False, description='Enable all debugging flags')
    variant('nls', default=True, description='Enable Native Language support')
    variant('openmp', default=False, description='Enable building with OpenMP support')
    variant('pvode-openmp', default=False, description='Enable building PVODE with OpenMP support')
    variant('python', default=True, description='Activate Python packages for data manipulation')
    variant('sundials', default=False, description='Use SUNDIALS CVODE, IDA, and ARKODE solvers')
    variant('blaslapack', default=False, description='Use BLAS and LAPACK')
    variant('petsc', default=False, description='Use PETSc interface')
    variant('slepc', default=False, description='Use SLEPc interface')
    variant('mumps', default=False, description='Use MUMPS library for direct matrix inversions')
    variant('scorep', default=False, description='Enable support for scorep based instrumentation')

    # Mandatory dependencies
    depends_on('autoconf', type='build')
    depends_on('automake', type='build')
    depends_on('mpi@3:', type=('build', 'link', 'run'))
    depends_on('fftw@3:')

    # Optional dependencies
    depends_on('hdf5 +mpi')
    depends_on('netcdf-c +mpi')
    depends_on('netcdf-cxx4')
    depends_on('python@3:', when='+python', type='run')
    depends_on('py-netcdf4', when='+python', type='run')
    depends_on('sundials +CVODE+IDA+ARKODE', when='+sundials')
    depends_on('netlib-lapack~external-blas', when='+blaslapack')
    depends_on('petsc@3.4.0:', when='+petsc')
    depends_on('slepc@3.4.0:', when='+slepc')
    depends_on('mumps', when='+mumps')
    depends_on('scorep', when='+scorep')

    def configure_args(self):
        spec = self.spec
        args = []

        if spec.satisfies('+shared'):
            args.append('--enable-shared')

        if spec.satisfies('checks=none'):
            args.append('--enable-checks=no')
        else:
            args.append('--enable-checks={}'.format(spec.variants['checks'].value))

        if spec.satisfies('optimize=none'):
            args.append('--enable-optimize=no')
        else:
            args.append('--enable-optimize={}'.format(spec.variants['optimize'].value))

        # Profiling and debugging
        if spec.satisfies('+track'):
            args.append('--enable-track')

        if spec.satisfies('+debug'):
            args.append('--enable-debug')

        if spec.satisfies('+scorep'):
            args.append('--with-scorep={}'.format(spec['scorep'].prefix.bin))

        # Native Language support
        if spec.satisfies('~nls'):
            args.append('--disable-nls')

        # OpenMP support
        if spec.satisfies('+openmp'):
            args.append('--enable-openmp')
        else:
            args.append('--disable-openmp')

        if spec.satisfies('+pvode-openmp'):
            args.append('--enable-pvode-openmp')

        # File format (only the parallel versions)
        #args.append('--with-hdf5={}/h5cc'.format(spec['hdf5'].prefix.bin))
        args.append('--with-parallelhdf5={}/h5pcc'.format(spec['hdf5'].prefix.bin))
        args.append('--with-netcdf={}'.format(spec['netcdf-cxx4'].prefix))
        #args.append('--with-pnetcdf={}'.format(spec['netcdf-cxx4'].prefix))

        # Math libraries
        args.append('--with-fftw={}'.format(spec['fftw'].prefix))

        if spec.satisfies('+sundials'):
            args.append('--with-sundials={}'.format(spec['sundials'].prefix))

        if spec.satisfies('+blaslapack'):
            args.append('--with-lapack={}'.format(spec['netlib-lapack'].prefix))

        if spec.satisfies('+petsc'):
            args.append('--with-petsc={}'.format(spec['petsc'].prefix))

        if spec.satisfies('+slepc'):
            args.append('--with-slepc={}'.format(spec['slepc'].prefix))

        if spec.satisfies('+mumps'):
            args.append('--with-mumps={}'.format(spec['mumps'].prefix))

        return args

With this configuration, commands for installing/uninstalling BOUT++ could be simplified to something like

$ spack install bout-dev +petsc+slepc+sundials+scorep+blaslapack checks=1 optimize=3

$ spack uninstall bout-dev

[ZedThree]

Thanks @alephpiece , this looks interesting! I've not used spack myself, but I know that it's widely used on clusters, supercomputers etc, so having a BOUT++ spack config would be useful.

Is this something we need to maintain in the BOUT++ repo ourselves? Or does the config file go in some official spack repo (like conda)?

[ZedThree]

Actually, I think the people at LLNL might have already done something with this -- @jonesholger is the spack stuff you've done specific to LLNL machines?

[alephpiece]

Thanks @alephpiece , this looks interesting! I've not used spack myself, but I know that it's widely used on clusters, supercomputers etc, so having a BOUT++ spack config would be useful.

Is this something we need to maintain in the BOUT++ repo ourselves? Or does the config file go in some official spack repo (like conda)?

Yes, it's just like conda. Spack has a builtin repo maintained by its contributors. I didn't find BOUT++ there so I have to create the package myself.

[alephpiece]

If there is already some BOUT++ configuration for Spack at some local repo, maybe it is time to push it to Spack? I could really use a fully tested Spack package to try BOUT++ on some machines.

[bendudson]

Some progress on Spack packaging for BOUT++: I've created a BOUT-spack repository: https://github.com/boutproject/BOUT-spack that we can use to host Spack packages. A difficulty is that BOUT-dev contains both the library and lots of examples. I don't know how those examples should be packaged.