Integration tests profiling

[GernotMaier]

Integration tests are a bit slow (in github: 25-35 min) and we might want to tune a bit:

52.94s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_north]
38.61s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_south]
35.71s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_corsika_corsika7_particle_id]
26.25s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_pack_for_south_grid]
22.28s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_north_pack_for_grid]
21.77s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_north_for_trigger_rates0]
20.12s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_north_for_trigger_rates1]
18.16s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_corsika]
17.75s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-simulate-prod_gamma_20_deg_corsika_pdg_particle_id]
9.99s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-derive-mirror-rnda_psf_mean]
6.60s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-generate-corsika-histograms_1d_run_header]
6.09s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-validate-camera-efficiency_MSTS]
5.92s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-derive-mirror-rnda_psf_measurement]
5.75s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-generate-corsika-histograms_individual_telescopes_and_indices]
5.74s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-validate-optics_run]
5.73s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-validate-camera-efficiency_SSTS]
5.57s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-validate-camera-efficiency_MSTN]
5.48s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-derive-mirror-rnda_psf_random_flen]
5.24s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-generate-corsika-histograms_1d_and_2d_run_header]
5.13s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-generate-simtel-array-histograms_hist_file_list]
5.13s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-validate-cumulative-psf_run]
5.04s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-derive-mirror-rnda_psf_no_tuning]
4.90s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-generate-corsika-histograms_2d_run_header]
4.81s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-generate-simtel-array-histograms_hist_file_names_pdf_hdf5]
4.70s call     tests/integration_tests/test_applications_from_config.py::test_applications_from_config[simtools-generate-simtel-array-histograms_hist_file_names_pdf_no_name]

Dominated by the testing of the production tool:

• currently 10 showers are simulated; this can safely dropped to 5
• should discuss if we need all those different tests: e.g.,
  • the only difference between simulate_prod_gamma_20_deg_North_corsika.yml, simulate_prod_gamma_20_deg_North_corsika_pdg_particle_id.yml, and simulate_prod_gamma_20_deg_North_corsika_corsika_particle_id.yml is that the particle ID is given in a different system. But this is tested in the corresponding unit tests (check!) and should not require to run a small set of test simulations
  • the only difference between simulate_prod_gamma_20_deg_North.yml and simulate_prod_gamma_20_deg_North_for_trigger_rates.yml is the one is running with gamma and other with proton. Is this really something we need to test?

(to be clear: we need to test as much as possible; but we won't learn from duplication)

[GernotMaier]

To make it clearer, this is the current test matrix:

Test	Software	Site	ParticleID	PackforGrid	Primary
simulate_prod_gamma_20_deg_North	corsika_simtel	North	simtools	No	gamma
simulate_prod_gamma_20_deg_North_corsika	corsika	North	simtools	No	gamma
simulate_prod_gamma_20_deg_North_corsika_corsika_particle_id	corsika	North	corsika	No	gamma
simulate_prod_gamma_20_deg_North_corsika_pdg_particle_id	corsika	North	PDG	No	gamma
simulate_prod_gamma_20_deg_North_for_trigger_rates	corsika_simtel	North	simtools	No	proton
simulate_prod_gamma_20_deg_North_pack_for_grid	corsika_simtel	North	simtools	Yes	gamma
simulate_prod_gamma_20_deg_South	corsika_simtel	South	simtools	No	gamma
simulate_prod_gamma_20_deg_South_for_trigger_rates	corsika_simtel	South	simtools	No	proton
simulate_prod_gamma_20_deg_South_pack_for_grid	corsika_simtel	South	simtools	Yes	gamma

I don't think this is necessary and my suggestion is to reduce it to:

Test	Software	Site	ParticleID	PackforGrid	Primary
simulate_prod_gamma_20_deg_North	corsika_simtel	North	simtools	Yes	gamma
simulate_prod_gamma_20_deg_North_corsika_corsika_particle_id	corsika	North	corsika	No	gamma
simulate_prod_gamma_20_deg_South_pack_for_grid	corsika_simtel	South	simtools	Yes	gamma
simulate_prod_gamma_20_deg_South_for_trigger_rates	corsika_simtel	South	PDG	No	proton

This would cover enough a broad range of combinations without too much duplication.

Feedback?

[GernotMaier]

@tobiaskleiner / @orelgueta - could you have a quick look at above suggestion to reduce the number of long integration tests? These are duplicated tests in my opinion, and we could add a bit of speed to the tests (which I really think is important for the development)

[tobiaskleiner]

@GernotMaier looks safe to remove these tests.