search

DUNE-DAQ / daq-release

Scripts and configuration files for the DUNE DAQ release
https://dune-daq-sw.readthedocs.io/en/latest/packages/daq-release/
2 stars 0 forks source link

JCF: allow the default-branch option for when all packages are checke… #289

Closed jcfreeman2 closed 1 year ago

jcfreeman2 commented 1 year ago

…d out in checkout-daq-package.py, permitting failures (e.g., you try to check out a prep-release branch and it's not there)

The idea is, if you want to look at the codebase corresponding to a manifest and want to prioritize a branch (e.g., a prep release branch) you can now do something like this:

./scripts/checkout-daq-package.py -i <manifest> -o <outputdir> -a -b <desired branch>

and as long as neither commit hashes nor versions are provided for the repos listed in the manifest, the script will try to check out the desired branch after cloning each repo. If there are commit hashes or versions listed, you can't use the -b argument, and instead the script will try to check out the commit or, if a commit isn't listed, the version.

For example, from the base of daq-release, if you do:

./scripts/checkout-daq-package.py  -i configs/fddaq/fddaq-v4.1.0/release.yaml -o  /tmp/fddaq_versions -a

then you'll get the far detector repos, checked out to their versions in v4.1.0.

If you do

./scripts/checkout-daq-package.py  -i configs/fddaq/fddaq-develop/release.yaml -o /tmp/fddaq_prep_release -a -b prep-release/dunedaq-v4.1.0

then you'll get the far detector repos, checked out to their v4.1.0 prep release branches if available, and develop otherwise.

If you do 

nightly_manifest=/cvmfs/dunedaq-development.opensciencegrid.org/nightly/NAFD23-07-20/spack-0.18.1-gcc-12.1.0/spack-0.18.1/spack-repo/NAFD23-07-20.yaml
./scripts/checkout-daq-package.py  -i $nightly_manifest -o /tmp/fddaq_nightly_july20 -a

the you'll get the codebase of the far detector nightly (this functionality already existed in the script).

dingp commented 1 year ago

LGTM.

There is an existing use case in one of the CI workflow for building a common feature branch across multiple repos together to form a special nightly release.

To do that, make-release-repo.py has a similar "branch overwrite" option when creating the spack repo, and capturing the commit hashes. It also regenerates the release yaml file with the captured commit hashes. So this new release yaml file contains the commit hashes of develop HEAD, and the feature branch HEAD wherever available. checkout-daq-package.py will use that regenerated release yaml file and checkout all of the DAQ repos accordingly. The end result is the same as running the script with the newly added option -b <desired branch> in most cases. But it was a two-step process.

The downside of using the new option is since the commit hashes are not captured, the checkout code may change between different runs in case of new pushes on the feature branches.

Maybe during future improvements, make-release-repo.py (should be renamed to something else as it can do many other things than creating the Spack-repo for a release, regenerating the release yaml with hashes is one of them), can incorporate the functionalities of checkout-daq-package.py, and capture the commit hash at the sometime for reproducibility.