Preparing for release 2.4

[alessandrothea]

1. Update python packages:

   • [x] Add pexpect
     • pexpect==4.8.0
     • ptyprocess==0.7.0
   • [x] Update moo: 0.5.6

2. Update external packages:

   • [x] move felix to products;
   • [x] build new version of felix v1_1_1 which only depends on gcc v8_2_0
   • [x] move pybind11 to products;
   • [x] move uhal to products (and its dependency package pugixml);
   • [x] rebuilt pugixml v1_11 with qualifier e19:prof, it now has proper pugixml::pugixml cmake target (previously broken);
   • create cmake config files for uhal and make a new version of it (post dunedaq-v2.4.0);

3. Update build order list

   • [x] with:

     set(build_order "daq-cmake" "ers" "logging" "cmdlib" "rcif" "restcmd" "opmonlib" "appfwk" "listrev" "daqdemos" "ipm" "serialization" "nwqueueadapters" "dataformats" "dfmessages" "dfmodules" "readout" "flxlibs" "trigemu" "influxopmon" "minidaqapp" "timing" "timinglibs")

4. Deploy daq-buildtools on CVMFS:

   • [x] put historical versions under sandbox/tools/dbt for test;
   • [x] move tools out of sandbox and add latest versions of dbt to it;

5. support for using new CMVFS repos (plan to switch in the next release, beta testing during 2.4)

   • [x] Copied over files under old repo to new repos #50;
   • [x] Deploy 2.4 to old cvmfs repo (default in dbt-settings.sh);
   • [x] Deploy 2.4 to new cvmfs repo;

6. Make release tag

   • [x] Tag packages included in this release with release tag dunedaq-v2.4.0.

List of DAQ packages and versions for this release:

repo	tag
daq-buildtools	v2.3.0
daq-cmake	v1.3.3
ers	v1.1.2
logging	v1.0.1 (advance UPS product version to v1.0.1b since v1.0.1 has been made with an older ers version)
cmdlib	v1.1.2
restcmd	v1.1.2
opmonlib	v1.1.0
rcif	v1.0.1 (advance UPS product version to v1.0.1b since v1.0.1 has been made with an older ers version)
appfwk	v2.2.2
listrev	v2.1.1 (advance UPS product version to v2.1.1b since v2.1.1 has been made with older versions of dependent packages)
serialization	v1.1.0
flxlibs	v1.0.0
dataformats	v2.0.0
dfmessages	v2.0.0
dfmodules	v2.0.2
trigemu	v2.1.0
readout	v1.2.0
minidaqapp	v2.1.1
ipm	v2.0.1
timing	v5.3.0
timinglibs	v1.0.0
influxopmon	v1.0.1
nwqueueadapters	v1.2.0

[strilov]

Hello. The pdtbutler tool in timing relies on the python package click_didyoumean. Could we please add it to the list of python tools to be installed?

Also, the timing and timinglibs still rely on the user to source an env.sh script (found in build/timing/scripts/env.sh, after building) before using the packages. This script checks that the required python packages are installed and that the pdtbutler can be sourced successfully. Crucially, it sets the environment variable PDT_TESTS to point to its default location $BASH_SOURCE)/../config/ which ends up as build/timing/config/ in a development environment. How do we handle this with the release and publishing to cvmfs?

[alessandrothea]

Alas we won't manage to have the support for the butler in this release. We'll look into it after 2.4.

[alessandrothea]

PDTS_TESTS, mhm. When running off UPS products, the equivalent is ${TIMING_SHARE}/config but I fear there is no similar variable when running from a local area.

[strilov]

From what I can see, the files in build/timing/config end up in install/timing/share/schema. Could we put them install/timing/share/config or install/timing/config? If so, then we can have an UPS product command which sets the PDT_TESTS to point the right place?

By a local area, do you mean when checking it out and building from source?

Also, I have a timing/python/CMakeLists.txt, which does the necessary to get the required files in the build and install directories to run pdtbutler, i.e.

[st15719@bayban timing_release_dev]$ find install/timing/lib64/python/
install/timing/lib64/python/
install/timing/lib64/python/timing
install/timing/lib64/python/timing/core
install/timing/lib64/python/timing/core/_core.so
install/timing/lib64/python/timing/core/__init__.py
install/timing/lib64/python/timing/core/src
install/timing/lib64/python/timing/__init__.py
install/timing/lib64/python/timing/shells
install/timing/lib64/python/timing/shells/master.py
install/timing/lib64/python/timing/shells/fmc.py
install/timing/lib64/python/timing/shells/tlu.py
install/timing/lib64/python/timing/shells/fanout.py
install/timing/lib64/python/timing/shells/boards.py
install/timing/lib64/python/timing/shells/__init__.py
install/timing/lib64/python/timing/shells/pc059.py
install/timing/lib64/python/timing/shells/factory.py
install/timing/lib64/python/timing/cli
install/timing/lib64/python/timing/cli/master.py
install/timing/lib64/python/timing/cli/endpoint.py
install/timing/lib64/python/timing/cli/debug.py
install/timing/lib64/python/timing/cli/io.py
install/timing/lib64/python/timing/cli/crt.py
install/timing/lib64/python/timing/cli/global.py
install/timing/lib64/python/timing/cli/__init__.py
install/timing/lib64/python/timing/cli/system.py
install/timing/lib64/python/timing/cli/click_texttable.py
install/timing/lib64/python/timing/cli/toolbox.py
install/timing/lib64/python/timing/cli/exttrig.py
install/timing/lib64/python/timing/cli/align.py
install/timing/lib64/python/timing/cli/definitions.py
install/timing/lib64/python/timing/common
install/timing/lib64/python/timing/common/database.py
install/timing/lib64/python/timing/common/__init__.py
install/timing/lib64/python/timing/common/definitions.py
install/timing/lib64/python/CMakeFiles
install/timing/lib64/python/CMakeFiles/_core.dir
install/timing/lib64/python/CMakeFiles/_core.dir/timing
install/timing/lib64/python/CMakeFiles/_core.dir/timing/core
install/timing/lib64/python/CMakeFiles/_core.dir/timing/core/src

I think that means we should be able to get pdtbutler working over cvmfs.

[strilov]

PDTS_TESTS, mhm. When running off UPS products, the equivalent is ${TIMING_SHARE}/config but I fear there is no similar variable when running from a local area.

If this is the case of building from source, then yes, indeed, setting PDT_TESTS will have to be a manual step. One idea could be a mechanism for packages to provide env.sh scripts, which are then called by something like dbt-setup-runtime-environment-user.

Or set it in the CMakeLists file of the package

[alessandrothea]

Sigh, yes, there's a bug in daq-cmake that makes config files end up in schema. Patch coming up.

Regarding pdtbutler, as I said, it's no-go for 2.4. I'm sorry about that but that other things came in the way. The building of the python bindings has to be disabled for the release and we will sort it out later (maybe in a patch release).

[alessandrothea]

PDTS_TESTS, mhm. When running off UPS products, the equivalent is ${TIMING_SHARE}/config but I fear there is no similar variable when running from a local area.

If this is the case of building from source, then yes, indeed, setting PDT_TESTS will have to be a manual step. One idea could be a mechanism for packages to provide env.sh scripts, which are then called by something like dbt-setup-runtime-environment-user.

Or set it in the CMakeLists file of the package

Let me rephease: PDS_TEST is deprecated. ${TIMING_SHARE}/config must be used in its place and daq-buildtools must have a way to provide it for development environments.

[dingp]

Additional items during beta testing period:

1. [x] add rich to pypi-repo;
2. [x] add rich to pyvenv_requirements.txt;
3. [x] update daq-buildtools in cvmfs for dunedaq-v2.4.0.

[dingp]

Release tag has been made. This concludes the release making process for dunedaq-v2.4.0.