add mwa packages

[d3v-null]

This was tested on Setonix on Spack 0.19 No module available to test on Spack 0.21

[dipietrantonio]

@ddeeptimahanti using the test partition, and after applying the above modification to the recipe, I managed to install all three packages. I noticed Spack chose to install hdf5@1.14.3 for some reason, I am looking into it.

[ddeeptimahanti]

@ddeeptimahanti using the test partition, and after applying the above modification to the recipe, I managed to install all three packages. I noticed Spack chose to install hdf5@1.14.3 for some reason, I am looking into it.

@dipietrantonio 1.14.3 will be the default hdf5 version in the new stack.

[dipietrantonio]

@dipietrantonio 1.14.3 will be the default hdf5 version in the new stack. @ddeeptimahanti

Yes, and given that is already installed, it should use the system installation but it doesn't, unless I specify the --reuse flag. May be because the spec metrix contains - [^libszip]?

[dipietrantonio]

@d3v-null As per the Python bindings, I am trying something like this

+++ b/repo/packages/mwalib/package.py
@@ -21,10 +21,14 @@ class Mwalib(Package):
     maintainers = ["d3v-null"]

     version("1.3.3", tag="v1.3.3")
+    variant("python", default=True, description="Build and install Python bindings.")

     depends_on("rust@1.64.0:", type="build")
     depends_on("cfitsio@3.49")
     depends_on("curl") # because cfitsio does not --disable-curl by default
+    depends_on("py-maturin", when="+python")
+    depends_on("py-numpy", when="+python")
+
     sanity_check_is_file = [
         join_path("include", "mwalib.h"),
         join_path("lib", "libmwalib.a"),
@@ -41,6 +45,7 @@ class Mwalib(Package):
     def install(self, spec, prefix):
         os.system("env")
         cargo = Executable("cargo")
+        maturin = which("maturin")
         with fs.working_dir(self.stage.source_path):
             cargo("build", "--release", "--features=cfitsio-static")
             shutil.copytree("include", f"{prefix}/include")
@@ -49,6 +54,8 @@ class Mwalib(Package):
             for f in release.iterdir():
                 if f.name.startswith("libmwalib."):
                     shutil.copy2(f"{f}", f"{prefix}/lib/")
+            maturin("build", "--release", "--features", "python,hdf5-static", "--strip")
+

     @run_after("install")
     @on_package_attributes(run_tests=True)

[ddeeptimahanti]

Yes @dipietrantonio , we enforced szip dependency due to some issue - @ilkhomab might still remember that one.

[d3v-null]

@dipietrantonio

looks good! my only comment is that you shouldn't do anything with maturin if the python feature is not enabled.

[dipietrantonio]

Hi Dev,

here is some progress on this. I had to modify the recipe for py-maturin to point the CARGO_HOME variable away from /home (see recipe below). This could be done in a nicer way directly in the rust recipe but I could not get it to work in a reasonable time, so this will do for now.

Then I could move on to try to build mwalib +python with the modified recipe (again, see below). Unfortunately I get the following error. Any idea?

1 error found in build log:
     559       Compiling regex v1.9.6
     560       Compiling toml v0.5.11
     561       Compiling mwalib v1.3.3 (/scratch/pawsey0001/cdipietrantonio/setonix/2024.05/software/cdipietrantonio/build_stage/spack-stage-mwalib-1.3.3-bziopg2cdwm2utgv4posvm2uqgusddvs/spack-src)
     562       Compiling fitsio v0.20.0
     563        Finished `release` profile [optimized] target(s) in 35.36s
     564    ==> [2024-06-05-20:50:43.195229] '/software/projects/pawsey0001/cdipietrantonio/setonix/2024.05/software/linux-sles15-zen3/gcc-12.2.0/py-maturin-1.1.0-mtv5birsrdlib7jqlkdyozhoffbozyfs/bin/maturin' 'b
            uild' '--release' '--features' 'python,hdf5-static' '--strip'
  >> 565    error: none of the selected packages contains these features: hdf5-static
     566    💥 maturin failed

py-maturin

# Copyright 2013-2023 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)

from spack.package import *

class PyMaturin(PythonPackage):
    """Build and publish crates with pyo3, rust-cpython and cffi bindings
    as well as rust binaries as python packages.
    """

    homepage = "https://github.com/pyo3/maturin"
    pypi = "maturin/maturin-0.13.7.tar.gz"

    version("1.1.0", sha256="4650aeaa8debd004b55aae7afb75248cbd4d61cd7da2dcf4ead8b22b58cecae0")
    version("0.14.17", sha256="fb4e3311e8ce707843235fbe8748a05a3ae166c3efd6d2aa335b53dfc2bd3b88")
    version("0.13.7", sha256="c0a77aa0c57f945649ca711c806203a1b6888ad49c2b8b85196ffdcf0421db77")

    depends_on("py-setuptools", type="build")
    depends_on("py-wheel@0.36.2:", type="build")
    depends_on("py-setuptools-rust@1.4:", type="build")
    depends_on("py-tomli@1.1:", when="^python@:3.10", type=("build", "run"))
    depends_on("rust", type=("build", "run"))

    def setup_build_environment(self, env):
        env.set("CARGO_HOME", f"{self.stage.source_path}/.cargo")

mwalib (diff)

@@ -21,10 +21,14 @@ class Mwalib(Package):
     maintainers = ["d3v-null"]

     version("1.3.3", tag="v1.3.3")
+    variant("python", default=True, description="Build and install Python bindings.")

     depends_on("rust@1.64.0:", type="build")
     depends_on("cfitsio@3.49")
     depends_on("curl") # because cfitsio does not --disable-curl by default
+    depends_on("py-maturin", when="+python")
+    depends_on("py-numpy", when="+python")
+
     sanity_check_is_file = [
         join_path("include", "mwalib.h"),
         join_path("lib", "libmwalib.a"),
@@ -49,6 +53,10 @@ class Mwalib(Package):
             for f in release.iterdir():
                 if f.name.startswith("libmwalib."):
                     shutil.copy2(f"{f}", f"{prefix}/lib/")
+            if '+python' in spec:
+                maturin = which("maturin")
+                maturin("build", "--release", "--features", "python,hdf5-static", "--strip")
+

     @run_after("install")
     @on_package_attributes(run_tests=True)

[d3v-null]

yup, mwalib doesn't have a feature hdf5-static, i think you probably want cfitsio-static

[d3v-null]

check out cargo.toml for a list of features. https://github.com/MWATelescope/mwalib/blob/33f900c589a40f02300e812db756e11d015a7711/Cargo.toml#L22