Closed andrewmogan closed 5 months ago
The core functionality looks good. One suggested improvement: if I scroll down at the output of https://github.com/DUNE-DAQ/daq-release/actions/runs/8209587413, what I see is the results of the full suite of tests printed out for each test. Could we eliminate this redundancy?
The redundant test summary output was caused by the xml parsing step being under the same job as the integration tests. Since the tests now run in a matrix strategy, the xml parsing was run separately for each integration test. The xml parsing step is now factored into its own job which only runs after all integration tests finish. See here.
Should be fixed now. The solution was just to move the if: always()
condition in the final step up in scope so that it runs even if the previous step fails. See here.
Output in https://github.com/DUNE-DAQ/daq-release/actions/runs/8299667779 looks like what we're shooting for; approved.
In response to Issue #351, this version of the nightly integration test workflow uses a matrix strategy to run each integration test in
daqsystemtest
and setsfail-fast: false
so that all tests will run even if one or more fail. The overall workflow will still show a failed state even if, for example, 3/4 succeed. This also makes it easier to add more integration tests in the future.Since the nightly actions are being shuffled around at the moment, in order to test this, I set
NIGHTLY_TAG: NFD_PROD4_240304_A9
and addedref: amogan/break_fake_data_test
underCheckout daqsystemtest
. This checks out a branch ofdaqsystemtest
where I've intentionally brokenfake_data_producer_test.py
. As desired, all tests ran,fake_data_producer_test
failed, and the overall workflow status showed that it failed.