Expt: choose Classifier thresholds for Infectious test , [Also fixes prevalence calculation]

[odunbar]

[Also fixes calculation of prevalence] We setup the test required to plot ROCs over a given experiment series for testing multiple % per day of infected. The idea is to generate the data once, then we can try different classifiers, metrics on the saved data easily. The plot tools I have written are to be used in conjunction with the way the outputs are stored in the batch script runs.

We have 4 files:

1. Generating and saving relevant data in backward_forward_assimilation.py (and all subsidiary _*.py files)

2. The batch script run_infectious_test_expt.sh runs backward_forward_assimilation.py for different infectious test configurations (different % per day)

3. The tool roc_from_savedata.py plots all ROC curves from the saved data. . Easily modifiable for different classifiers, taken at different times in the series (currently at the end times)

4. The tool tprplot_from_savedata.py plots all True Positive Rate vs Predicted Positive Fraction graphs from the save data. Same interface as with 3.

Output ROCs for an experiment (30 day run, 1000 nodes, 12 day spinup, 5 day DA sweep, 99% sensitivity, 99% specificity (noisy_measurement=True)) the graph markers indicate different threshold values (though many threshold values may take the same value)

Output TPR vs PPF plots for the experiment

[odunbar]

OK - So I have also incorporated Dima's latest PR with some spring cleaning:

The workhorse file is called:

• backward_forward_assimilation.py. The file true_rates_estimation.py has been removed due to overlap of purpose.

There are 2 experiments with sbatch / script files, to call the above with in a job array with different parameters

• run_infectious_test_expt (i have kept run_infectious_test_expt_parallel separate, as it require large resources compared to the serial version, so only to be used if necessary)
• run_user_base_expt (formerly true_rates_estimation_sbatch/ script)

I haven't changed the internals of the scripts so these should work as they did prior to the merging.

There are 3 postprocess files, which will run exactly on the stored outputs from a run_infectious_test_expt.sh

• roc_from_savedata.py
• tprplots_from_savedata.py
• plot_monotone_infectiousness.py

These are all useful for assessing the efficacy of our methods.