Closed andrewmogan closed 5 months ago
Forgot to reference Issue #139
Thanks a lot. This overall looks good, but there are three things to address:
1) While it's currently the case that far-detector-specific packages and near-detector-specific packages get the benefits of the new algorithm, it appears that this is not the case for the common packages (e.g. appfwk, ipm, etc.)
2) When get_cmake_dependencies
is called, regardless of the desired branch of the package in question, it's always the CMakeLists.txt
file from the develop
branch which gets parsed
3) If a failure to curl
a CMakeLists.txt
file occurs, currently this parses as "print a warning and give the package in question no dependencies". We should instead treat this as a catastrophic error and have the script exit at this point; see this article by software guru Martin Fowler on the principle behind this advice.
Now we're getting to fine tuning things. I did a side-by-side comparison of all the package.py
files both using the traditional, "hardwired" package.py
s and the package.py
s generated by this script. The following need to be addressed, and once this is done we should be good to go:
#find_package(Boost ...
in appdal
's CMakeLists.txt
results in the addition of depends_on("boost")
in the package.py
file. devtools
doesn't depend on systems
, and externals
doesn't depend on devtools
in the package.py
files which appear; this should be restored. hdf5libs
depends on highfive
, and the script correctly picks up on this. However, not unlike the situation we ran into with the folly
dependency, we need to make sure that highfive
is built with MPI support, i.e. depends_on("highfive +mpi")
even though this fact isn't provided in CMakeLists.txt
find_package(felix ....
, a couple more packages don't have names which are precisely referred to in CMakeLists.txt
. Mapping their CMakeLists.txt
names to their Spack names, they are:
PkgConfig
-> pkgconf
(from readoutlibs
)TBB
-> intel-tbb
(from cmdlib
)CMakeLists.txt
file. In these situations, we should just hardwire the dependencies into the template package.py
just below the XDEPENDSX
token. Specifically:
daq-cmake
should have a dependency on py-pybind11
. The reasoning behind this is that even though py-pybind11
isn't used to build daq-cmake
itself, packages which use daq-cmake
may need to use it for Python bindings. erskafka
should have dependencies on cyrus-sasl
and openssl
. To be honest I can't recall the original motivation for this; until I figure it out we should have them in there based on the principle of Chesterton's fenceWith the current head of the feature branch, I successfully created a test nightly for the v4.3.0 release (https://github.com/DUNE-DAQ/daq-release/actions/runs/8270039867) and ran minimal_system_quick_test.py
with it both on daq.fnal.gov
and np04-srv-015
. Approved.
This PR modifies
make-release-repo.py
and thepackage.py
files for both FD and ND packages. Thedepends_on("package")
list in the spack recipes are replaced byXDEPENDSX
which, similar toXVERSIONX
andXHASHX
, will be replaced when generating the package repository. Basically,DAQRelease.generate_daq_package
will now parse each package'sCMakeLists.txt
and infer dependencies fromfind_package
calls. Some exceptions are handled with brute forceif/elif
statements. The Alma9 nightly build succeeds when using this branch, and I can successfully create a local area based on this branch from cvmfs.