Indicators Proposed by Mehran for Summarising Impact Do Not Behave as Expected In IC7

[tbhallett]

In the course of responding to #30, @mehrhos proposes some indicators for measuring progress toward the GP targets that is achieved under a particular funding scenario: https://github.com/globalfund-pri/model_code_pipeline/pull/15#issuecomment-2238514920

In particular:

I recall from last time that we wanted to know what would be the impact of these choices on % reduction on incidence, mortality, cases-infections and deaths from baseline to the last year of IC (i.e. from 2023 to 2029). I wonder if we could have a version of these graphs where y axis is that metric, i.e. showing the ratio of % reduction from 2023 to 2029 according to IC vs. same % according to GP, e.g. with 18bn GF funding applying approach B including unallocated and adjusting for innovation, we would reduce number of deaths from 2023 to 2029 by 60% compared to 80% in GP, so the % impact achieved is 60% / 80% = 75%.

Looking into this, reveals that these indicators do not behave as expected......

.. essentially because

• none of the optimisation (at country level or in Approach B) is optimising to match the GP reduction in 2030. (They are optimising to minimise cases and death over a period.)
• 2022 is different between the scenarios and GP

This may be solved with the additional variations on GP being requested in IC8.

[tbhallett]

I used the following script to investigate this issue:

"""Investigate how the indicators Mehran has proposed perform."""
from matplotlib import pyplot as plt

from scripts.ic7.hiv.hiv_filehandlers import ModelResultsHiv
from scripts.ic7.malaria.malaria_filehandlers import ModelResultsMalaria
from scripts.ic7.tb.tb_filehandlers import ModelResultsTb
from tgftools.filehandler import Parameters
from tgftools.utils import get_root_path, get_data_path, save_var

path_to_data_folder = get_data_path()
project_root = get_root_path()

# Declare the parameters, indicators and scenarios
parameters = Parameters(project_root / "src" / "scripts" / "ic7" / "shared" / "parameters.toml")

# Load modelling results for HIV, TB and Malaria
model_results = {
    'hiv': ModelResultsHiv(
        path_to_data_folder / "IC7/TimEmulationTool/modelling_outputs/hiv",
        parameters=parameters
    ),
    'tb': ModelResultsTb(
        path_to_data_folder / "IC7/TimEmulationTool/modelling_outputs/tb",
        parameters=parameters),
    'malaria': ModelResultsMalaria(
        path_to_data_folder / "IC7/TimEmulationTool/modelling_outputs/malaria/standard",
        parameters=parameters)
}
save_var(model_results, project_root / 'sessions' / 'model_results.pkl')

for disease, model_result in model_results.items():
    df = model_result.df
    df.index.names  # FrozenList(['scenario_descriptor', 'funding_fraction', 'country', 'year', 'indicator'])

    for indicator in ('cases', 'deaths'):
        # Get the time-trend for this indicator
        time_trend = df.loc[('IC_IC', slice(None), slice(None), slice(None), indicator), 'central'].groupby(['year', 'funding_fraction']).sum().unstack()

        # Compute reduction from 2022 to 2029 for each funding_fraction
        reduction = 100 * (1. - time_trend.loc[2029]/time_trend.loc[2022])

        # Express the reduction acheived under each funding_fraction as a percentage of the reduction achieved under full_funding
        reduction_vs_ff = 100 * (reduction / reduction.at[1.00])

        # Plot:
        fig, ax = plt.subplots(ncols=2, nrows=1, sharey=False)
        time_trend.plot(ax=ax[0])
        ax[0].set_title('Time Trend')
        reduction_vs_ff.plot(ax=ax[1])
        ax[1].set_title('Reduction 2023-20239 \nas % of That Under Full Funding')
        fig.suptitle(f"{disease}: {indicator}")
        fig.savefig()
        fig.tight_layout()
        fig.savefig(get_root_path() / 'outputs' / f"progress_twd_gp_{disease}: {indicator}.png")
        fig.show()
        plt.close(fig)

[tbhallett]

I'm closing this. I think, on reflection, it does behave as expected. I was just expecting the wrong thing.