search

LSSTDESC / RESSPECT

The RESSPECT project is a result from an inter-collaboration agreement established between the Cosmostatistics Initiative (COIN) and the LSST Dark Energy Science Collaboration (DESC) with the goal of developing a recommendation system for telescope resource allocation able to optimize photometric supernova cosmology anaylsis.
MIT License
1 stars 0 forks source link

Determine best path forward for end-to-end integration test `run-spncc-e2e.sh` #9

Closed drewoldag closed 2 months ago

drewoldag commented 2 months ago

One of the CI scripts, .github/workflows/e2e-tests.yml will execute a bash shell script that does an end to end run of the run_loop command line arg.

To do so it has to copy and extract some SPNCC data, define a bunch of output locations, and then ultimately prints out some metrics.

Given that there is no test on the accuracy of the results, it's not immediately clear how much value this adds.

Either way, we should determine how this will be executed by the PPT CI workflows. And how to incorporate it into the pre-commit execution step as well.

drewoldag commented 2 months ago

Looking over this test, a large part of it is not really testing anything. It's mostly decompressing a zip file and moving data around. I would recommend that we do the following. It does exercise a couple of the CLI scripts, and an argument could be made that it's checking the argparser for those as well.

~1) Extract a small subset of the data into a data/test/ directory (similar to what we've done in .../data/test/DES_data so that we don't have to extract and move data for every test instance.~ This isn't required, because we can reuse the same data that the other test in test_learn_loop uses.

2) Create another test in test_learn_loop,py that will execute the same code with different config parameters.

3) Set up a different test that will verify that CLI args are being parsed correctly for the various command line scripts.