Proposal for PDVD CI test

[YoannKermaidic]

Suggestion to add PDVD CI test for the sim/reco. Reasons are:

• the chain broke in previous release.
• software skeleton for PDVD is converging to a rather stable state.
• as MC productions are coming, need to easily pin point bugs during the development phase

Ongoing discussion with Vito Di Benedetto. Do not merge before green light.

[YoannKermaidic]

Successfully test the PDVD ci test with a single 2 GeV muon crossing the cryostat:

-bash-4.2$ test_runner -O[DEFAULT]ignore_skip=1 -v seq_test_protoDUNEvd
Found config: /cvmfs/fermilab.opensciencegrid.org/products/common/db/../prd/generic_ci/v2_1_1/NULL/test/ci_tests.cfg
Found config: /dune/app/users/ykermaid/sw/dune_workdir/localProducts_larsoft_v09_82_02d01_prof_e26/dunesw/v09_82_02d01/test/ci_tests.cfg
The current parallel limit is: 5
Test Suite: seq_test_protoDUNEvd
Expanded: ci_gen_regression_seq_test_protoDUNEvd ci_g4_s1_regression_seq_test_protoDUNEvd ci_g4_s2_regression_seq_test_protoDUNEvd ci_detsim_regression_seq_test_protoDUNEvd ci_reco_regression_seq_test_protoDUNEvd
      Start  1: ci_gen_regression_seq_test_protoDUNEvd
 1/ 5 Test  #1: ci_gen_regression_seq_test_protoDUNEvd ..  Passed    2.34 sec;   10.76 kVs; 323684.00 kb; 
      Start  2: ci_g4_s1_regression_seq_test_protoDUNEvd
 2/ 5 Test  #2: ci_g4_s1_regression_seq_test_protoDUNEvd ..  Passed   11.62 sec;   53.45 kVs; 461508.00 kb; 
      Start  3: ci_g4_s2_regression_seq_test_protoDUNEvd
 3/ 5 Test  #3: ci_g4_s2_regression_seq_test_protoDUNEvd ..  Passed   59.04 sec;  271.59 kVs; 574328.00 kb; 
      Start  4: ci_detsim_regression_seq_test_protoDUNEvd
 4/ 5 Test  #4: ci_detsim_regression_seq_test_protoDUNEvd ..  Passed   60.07 sec;  276.33 kVs; 1592764.00 kb; 
      Start  5: ci_reco_regression_seq_test_protoDUNEvd
 5/ 5 Test  #5: ci_reco_regression_seq_test_protoDUNEvd ..  Passed  186.93 sec;  859.89 kVs; 2600076.00 kb; 

5 tests passed (100%), 0 tests failed, 0 tests with warnings, 0 tests skipped, out of 5
Not updating any reference files

[YoannKermaidic]

Reference files have been put into : /pnfs/dune/persistent/stash/ContinuousIntegration/protoDUNEvd gen g4_s1 g4_s2 detsim reco

[YoannKermaidic]

New branch created in lar_ci to accomodate these new tests: https://cdcvs.fnal.gov/redmine/projects/lar_ci/repository?utf8=%E2%9C%93&rev=feature%2Fkermaidic_pdvdci

[YoannKermaidic]

@vitodb, can you please take a look at and check this PR?

[vitodb]

@YoannKermaidic the CI configuration part for this PR looks good. Checking the branch in lar_ci it looks good to, though, I think, there was an unintended edit in cfg/workflow_DUNE.cfg at line 159 where seq_test_protoDUNEsp has been replaced by seq_testdprotoDUNEsp. But other than that all looks good. I see on dCache there is a set of references/input for the new CI test suite in /pnfs/dune/persistent/stash/ContinuousIntegration/protoDUNEvd/. I would advice to make all folders and references files in there group writable, so in the future if someone else from DUNE needs to update references, they can do it.

[vitodb]

@YoannKermaidic with reference files in place, you can make another test of the new CI test suite to check those references are good. For this purpose, once you setup your dunesw code for this PR, setup lar_ci, then run: test_runner -v -s quick_test_protoDUNEvd

[YoannKermaidic]

@YoannKermaidic the CI configuration part for this PR looks good. Checking the branch in lar_ci it looks good to, though, I think, there was an unintended edit in cfg/workflow_DUNE.cfg at line 159 where seq_test_protoDUNEsp has been replaced by seq_testdprotoDUNEsp. But other than that all looks good. I see on dCache there is a set of references/input for the new CI test suite in /pnfs/dune/persistent/stash/ContinuousIntegration/protoDUNEvd/. I would advice to make all folders and references files in there group writable, so in the future if someone else from DUNE needs to update references, they can do it.

Lar_ci typo fixed and permissions updated on dCache.

[YoannKermaidic]

@YoannKermaidic with reference files in place, you can make another test of the new CI test suite to check those references are good. For this purpose, once you setup your dunesw code for this PR, setup lar_ci, then run: test_runner -v -s quick_test_protoDUNEvd

Test done. This requested updates in the config threshold and pointed to a bug in the detsim stage.

[YoannKermaidic]

-bash-4.2$ test_runner -O[DEFAULT]ignore_skip=1 -v seq_test_protoDUNEvd
Found config: /cvmfs/fermilab.opensciencegrid.org/products/common/db/../prd/generic_ci/v2_1_1/NULL/test/ci_tests.cfg
Found config: /dune/app/users/ykermaid/sw/dune_workdir/localProducts_larsoft_v09_82_02d01_prof_e26/dunesw/v09_82_02d01/test/ci_tests.cfg
The current parallel limit is: 5
Test Suite: seq_test_protoDUNEvd
Expanded: ci_gen_regression_seq_test_protoDUNEvd ci_g4_s1_regression_seq_test_protoDUNEvd ci_g4_s2_regression_seq_test_protoDUNEvd ci_detsim_regression_seq_test_protoDUNEvd ci_reco_regression_seq_test_protoDUNEvd
      Start  1: ci_gen_regression_seq_test_protoDUNEvd
 1/ 5 Test  #1: ci_gen_regression_seq_test_protoDUNEvd ..  Passed    2.34 sec;   10.76 kVs; 323660.00 kb; 
      Start  2: ci_g4_s1_regression_seq_test_protoDUNEvd
 2/ 5 Test  #2: ci_g4_s1_regression_seq_test_protoDUNEvd ..  Passed   10.62 sec;   48.85 kVs; 464500.00 kb; 
      Start  3: ci_g4_s2_regression_seq_test_protoDUNEvd
 3/ 5 Test  #3: ci_g4_s2_regression_seq_test_protoDUNEvd ..  Passed   67.06 sec;  308.48 kVs; 622296.00 kb; 
      Start  4: ci_detsim_regression_seq_test_protoDUNEvd
 4/ 5 Test  #4: ci_detsim_regression_seq_test_protoDUNEvd ..  Passed   42.14 sec;  193.85 kVs; 1428700.00 kb; 
      Start  5: ci_reco_regression_seq_test_protoDUNEvd
 5/ 5 Test  #5: ci_reco_regression_seq_test_protoDUNEvd ..  Passed  106.12 sec;  488.16 kVs; 2371696.00 kb; 

5 tests passed (100%), 0 tests failed, 0 tests with warnings, 0 tests skipped, out of 5
Not updating any reference files

[YoannKermaidic]

-bash-4.2$ test_runner -v -s quick_test_protoDUNEvd
Found config: /cvmfs/fermilab.opensciencegrid.org/products/common/db/../prd/generic_ci/v2_1_1/NULL/test/ci_tests.cfg
Found config: /dune/app/users/ykermaid/sw/dune_workdir/localProducts_larsoft_v09_82_02d01_prof_e26/dunesw/v09_82_02d01/test/ci_tests.cfg
The current parallel limit is: 5
Test Suite: quick_test_protoDUNEvd
Expanded: ci_gen_regression_test_protoDUNEvd ci_g4_s1_regression_test_protoDUNEvd ci_g4_s2_regression_test_protoDUNEvd ci_detsim_regression_test_protoDUNEvd ci_reco_regression_test_protoDUNEvd
      Start  1: ci_gen_regression_test_protoDUNEvd
      Start  2: ci_g4_s1_regression_test_protoDUNEvd
      Start  3: ci_g4_s2_regression_test_protoDUNEvd
      Start  4: ci_detsim_regression_test_protoDUNEvd
      Start  5: ci_reco_regression_test_protoDUNEvd
[...]
 1/ 5 Test  #1: ci_gen_regression_test_protoDUNEvd ..  Passed    2.23 sec;   10.26 kVs; 323684.00 kb; 
[...]
 2/ 5 Test  #2: ci_g4_s1_regression_test_protoDUNEvd ..  Passed   11.46 sec;   52.72 kVs; 470860.00 kb; 
[...]
 3/ 5 Test  #3: ci_detsim_regression_test_protoDUNEvd ..  Passed   41.63 sec;  191.50 kVs; 1435932.00 kb; 
[...]
 4/ 5 Test  #4: ci_g4_s2_regression_test_protoDUNEvd ..  Passed   60.11 sec;  276.51 kVs; 933952.00 kb; 
[...]
 5/ 5 Test  #5: ci_reco_regression_test_protoDUNEvd ..  Passed  121.40 sec;  558.45 kVs; 2900852.00 kb; 

5 tests passed (100%), 0 tests failed, 0 tests with warnings, 0 tests skipped, out of 5
Not updating any reference files

[vitodb]

All looks good!

[tomjunk]

trigger build with DUNE/dunesw#102

[FNALbuild]

:heavy_check_mark: CI build for LArSoft Succeeded on slf7 for e26:prof -- details available through the CI dashboard

[FNALbuild]

:heavy_check_mark: CI build for LArSoft Succeeded on slf7 for c14:prof -- details available through the CI dashboard

[FNALbuild]

:x: CI build for DUNE Failed at phase build DUNE on slf7 for e26:prof -- details available through the CI dashboard

:rotating_light: For more details about the failed phase, check the build DUNE phase logs

parent CI build details are available through the CI dashboard

[FNALbuild]

:x: CI build for DUNE Failed at phase build DUNE on slf7 for c14:prof -- details available through the CI dashboard

:rotating_light: For more details about the failed phase, check the build DUNE phase logs

parent CI build details are available through the CI dashboard

[tomjunk]

trigger build with DUNE/dunesw#102