Creating Automated Testing Pipeline

[parsotat]

There are a few things that we need to think about in order to start implementing automated testing. These are:

1. What are the things that we need to test? We definitely want to test the llh calculation and the bkg estimation, what else?
2. Can we create any codes that are self-contained to test the points in item 1?
3. What GRB do we want to make the default test case that we test everything against?

[Tohuvavohu]

Including @jjd330

1. We want to test, at minimum, the response construction, forward model, and likelhood evaluation. The bkg estimation is probably a good idea too..
2. We can probably convert the https://github.com/Swift-BAT/NITRATES/blob/main/Example_LLH_setup_fixed_dirs.ipynb notebook into a self-contained code for the tests, but this does not currently include the bkg fit (it just reads it in). How easy to include this?
3. We can use the GRB that's in the above notebook, and I think we had agreed on also having an OFOV burst? or just testing the likelihood using an OFOV position for this burst? Do you remember?

[parsotat]

1. I agree that we should include the background. You never know where a bug might pop up and the background estimation is a pretty important part of the pipeline.
2. @jjd330 ?
3. If I remember correctly, we wanted 2 test cases of an IFOV burst and an OFOV burst.

[jjd330]

1. Do we just want to run the bkg estimation script on the test data as the test or do we want just part of the bkg script?
2. It doesn't include fitting the bkg, but it uses the bkg models (diffuse and known sources)
3. Yes, we decided to use an OFOV burst too. I picked a position at some point to do, that I included the responses for in what I sent to @Tohuvavohu . I'll have to look through old messages to find the exact position.

[Tohuvavohu]

1. Probably actually run the bkg estimation script, unless you think all functionality can be tested with a subset?
2. Right.
3. Right, ok. Unfortunately Slack has now paywalled our old messages so I can't find this, or the pre-prepared responses you made for both positions.

[jjd330]

The position I decided to use was theta, phi = 125 deg, 25 deg. Here's the response files I shared on slack before https://alabama.box.com/s/n5s2h18nthyaiplrp917ygv07leqz173

[Tohuvavohu]

Thanks Jimmy, and the response for the ifov pos?

[jjd330]

They're all in that tar file, the inFoV and outFoV responses necessary for the tests.

[Tohuvavohu]

@metalphase lets discuss and develop the pytest framework here

[Tohuvavohu]

There is a first PR from @metalphase for the test framework here: https://github.com/parsotat/NITRATES/pull/2

[Tohuvavohu]

tests of the sqrt(TS) calculation for both IFOV and OFOV are done and in the /tests directory. do we also want a test for the bkg fit?

[Tohuvavohu]

Bumping this. Do we want an automated test of the bkg fit? or of any of the actual search scripts?