What is the "Correct" Mode of HR Constraints in the Model (Overall or for different analyses)

[tbhallett]

@marghe-molaro, @BinglingICL and @tbhallett have discussed what is the "correct" mode of the HR constraints in the model for various purposes.

The provisional agreement is as follows:

• To see if for all purposes we can align around a version of the HR constraints that is a mixture of Mode 2 (Hard Constraints) and Mode 1 (Elastic Constraints):

  • Define for each HSI a maximum tolerated squeeze-factor, _Sh
  • If the actual squeeze-factor for that HSI, S <= _Sh : then run the HSI as normal (as per elastic constraints)
  • But, if S > _Sh , then do not run the HSI (shunt is the next day), as per Hard constraints.
  • And this should be run with referrals being as high as indicated in the clinical guidelines (per #865) .... this would be suitable for all applications of the model and would mean that all model applications would have the same epidemiological outcomes (for the same HR and epidemiology assumptions.) This also has the advantage of being (perhaps) the most realistic / defensible form of assumption. BUT, it may disturb the epidemiological calibrations and this will need to be investigated.

• And, if not.... then:

  • Use Mode = 1 (Elastic Constraints) for analyses in which the HR constraint (and changes to it) are not important: e.g. most disease-specific analyses, Bingling's description of HR staff being "over-loaded".
  • Use Mode = 2 (Hard Constraints) for analyses in which the HR constraints (and changes to it) are important: e.g. estimating the value of increasing HCW time, and finding the optimal set of TREATMENT_ID to allow for a given configuration of the health system. ... and accepting that there will be difference in the epidemiological outcomes of between these two modes. (This is basically the approach we're adopting by default at the moment).

[A related issue is whether the squeeze-factor should apply progressively to things with lower priority.]

Next steps are: 1) Examine what squeeze-factors are actually applying to HSI in the current form of the model under Mode=1 (@tbhallett will do) 2) Examine the different in health-outcomes for Mode=2 vs Mode=1 (@tbhallett will do) 3) Discuss further and then..... 4) Experiment with introducing the new "Mode=4"....

[tbhallett]

What are the Squeeze-Factors in the Current Model?

I looked at this on the branch PR’d at #787 in src/scripts/calibration_analyses/analysis_scripts/analysis_hsi_descriptions.py:figure7_squeeze_factors

Results are shown here.

We note that almost all HSI have an average squeeze_factor > 0.0

The only ones that have 0.0 are: Inpatient_Care -- and this is by design (if you're already an in-patient, that appointment has to happen).

TREATMENT_ID:HSI	Squeeze_Factor
Inpatient_Care:Inpatient_Care	0
Epilepsy_Treatment_Followup:HSI_Epilepsy_Follow_Up	137.579193
Hiv_Treatment:HSI_Hiv_StartOrContinueTreatment	16.4579925
Tb_Test_FollowUp:HSI_Tb_FollowUp	11.9069271
Hiv_Test:HSI_Hiv_TestAndRefer	3.48464806
Alri_Pneumonia_Treatment_Inpatient_Followup:HSI_Alri_Treatment	17.4918132
FirstAttendance_NonEmergency:HSI_GenericFirstApptAtFacilityLevel0	2.77036774
PostnatalCare_Neonatal:HSI_NewbornOutcomes_ReceivesPostnatalCheck	18.5985309
Rti_FractureCast:HSI_RTI_Fracture_Cast	55.5153206
Rti_MinorSurgeries:HSI_RTI_Minor_Surgeries	34.544554
Rti_MajorSurgeries:HSI_RTI_Major_Surgeries	36.9783094
Rti_Suture:HSI_RTI_Suture	15.3722257
Rti_OpenFractureTreatment:HSI_RTI_Open_Fracture_Treatment	20.8343863
Rti_AcutePainManagement:HSI_RTI_Acute_Pain_Management	14.6304034
DeliveryCare_Basic:HSI_Labour_ReceivesSkilledBirthAttendanceDuringLabour	13.8789467
Tb_Test_Screening:HSI_Tb_ScreeningAndRefer	17.7852903
FirstAttendance_Emergency:HSI_GenericEmergencyFirstApptAtFacilityLevel1	55.5021346
Depression_TalkingTherapy:HSI_Depression_TalkingTherapy	51.274745
CardioMetabolicDisorders_Treatment:HSI_CardioMetabolicDisorders_Refill_Medication	17.9647589
Hiv_Prevention_Infant:HSI_Hiv_StartInfantProphylaxis	20.7143948
Contraception_Routine:HSI_Contraception_FamilyPlanningAppt	11.6179009
Undernutrition_Feeding:HSI_Stunting_ComplementaryFeeding	17.3307212
CardioMetabolicDisorders_Investigation:HSI_CardioMetabolicDisorders_InvestigationNotFollowingSymptoms	21.5128789
OesophagealCancer_Investigation:HSI_OesophagealCancer_Investigation_Following_Dysphagia	29.8771417
Measles_Treatment:HSI_Measles_Treatment	19.5650175
PostnatalCare_Neonatal_Inpatient:HSI_NewbornOutcomes_NeonatalWardInpatientCare	24.5031878
Diarrhoea_Treatment_Inpatient:HSI_Diarrhoea_Treatment_Inpatient	19.4199343
Epilepsy_Treatment_Start:HSI_Epilepsy_Start_Anti_Epileptic	141.187826
FirstAttendance_SpuriousEmergencyCare:HSI_EmergencyCare_SpuriousSymptom	18.7227554
Rti_MedicalIntervention:HSI_RTI_Medical_Intervention	90.0375758
Malaria_Treatment_NotComplicated_Child:HSI_Malaria_non_complicated_treatment_age0_5	284.995131
Alri_Pneumonia_Treatment_Outpatient:HSI_Alri_Treatment	12.5218939
AntenatalCare_Outpatient:HSI_CareOfWomenDuringPregnancy_FirstAntenatalCareContact	16.097668
BreastCancer_Treatment:HSI_BreastCancer_PostTreatmentCheck	0.04361475
AntenatalCare_Outpatient:HSI_CareOfWomenDuringPregnancy_SecondAntenatalCareContact	25.8052423
AntenatalCare_FollowUp:HSI_CareOfWomenDuringPregnancy_AntenatalOutpatientManagementOfAnaemia	36.7931626
Rti_Imaging:HSI_RTI_Imaging_Event	15.5710707
PostnatalCare_Maternal:HSI_Labour_ReceivesPostnatalCheck	15.673397
DeliveryCare_Neonatal:HSI_NewbornOutcomes_CareOfTheNewbornBySkilledAttendantAtBirth	19.924876
Malaria_Test:HSI_Malaria_rdt	494.445905
AntenatalCare_Inpatient:HSI_CareOfWomenDuringPregnancy_AntenatalWardInpatientCare	23.0120154
Diarrhoea_Treatment_Outpatient:HSI_Diarrhoea_Treatment_Outpatient	18.6263261
Tb_Test_Xray:HSI_Tb_Xray_level1b	14.3746146
ProstateCancer_Investigation:HSI_ProstateCancer_Investigation_Following_Urinary_Symptoms	29.6449212
Schisto_Treatment:HSI_Schisto_TestingFollowingSymptoms	13.6940662
Malaria_Treatment_NotComplicated_Adult:HSI_Malaria_non_complicated_treatment_adult	328.565947
Malaria_Treatment_NotComplicated_Child:HSI_Malaria_non_complicated_treatment_age5_15	355.018251
ProstateCancer_Investigation:HSI_ProstateCancer_Investigation_Following_psa_positive	38.9784987
Schisto_Treatment:HSI_Schisto_TreatmentFollowingDiagnosis	14.754001
Alri_Pneumonia_Treatment_Inpatient:HSI_Alri_Treatment	19.1122444
ProstateCancer_Treatment:HSI_ProstateCancer_StartTreatment	1.7288691
Malaria_Prevention_Iptp:HSI_MalariaIPTp	3.90974815
Epi_Pregnancy_Td:HSI_TdVaccine	3.26813791
CardioMetabolicDisorders_Prevention_WeightLoss:HSI_CardioMetabolicDisorders_StartWeightLossAndMedication	24.9027379
OesophagealCancer_Treatment:HSI_OesophagealCancer_StartTreatment	1.8666144
Depression_Treatment:HSI_Depression_Refill_Antidepressant	9.68692328
CardioMetabolicDisorders_Prevention_CommunityTestingForHypertension:HSI_CardioMetabolicDisorders_CommunityTestingForHypertension	18.4738973
CardioMetabolicDisorders_Investigation:HSI_CardioMetabolicDisorders_InvestigationFollowingSymptoms	23.3677304
CardioMetabolicDisorders_Treatment:HSI_CardioMetabolicDisorders_SeeksEmergencyCareAndGetsTreatment	5.15292496
Depression_Treatment:HSI_Depression_Start_Antidepressant	84.6139559
AntenatalCare_PostEctopicPregnancy:HSI_CareOfWomenDuringPregnancy_TreatmentForEctopicPregnancy	31.4237515
DeliveryCare_Comprehensive:HSI_Labour_ReceivesComprehensiveEmergencyObstetricCare	18.9839791
Hiv_Prevention_Circumcision:HSI_Hiv_Circ	19.2844506
Tb_Treatment:HSI_Tb_StartTreatment	16.7440051
PostnatalCare_Maternal_Inpatient:HSI_Labour_PostnatalWardInpatientCare	25.8331296
AntenatalCare_Outpatient:HSI_CareOfWomenDuringPregnancy_ThirdAntenatalCareContact	1.83089288
AntenatalCare_Outpatient:HSI_CareOfWomenDuringPregnancy_FourthAntenatalCareContact	2.15203702
AntenatalCare_Outpatient:HSI_CareOfWomenDuringPregnancy_FifthAntenatalCareContact	2.56641501
Tb_Test_Xray:HSI_Tb_Xray_level2	15.1288803
OtherAdultCancer_Investigation:HSI_OtherAdultCancer_Investigation_Following_early_other_adult_ca_symptom	24.3897215
AntenatalCare_Inpatient:HSI_CareOfWomenDuringPregnancy_MaternalEmergencyAssessment	12.9751508
AntenatalCare_PostAbortion:HSI_CareOfWomenDuringPregnancy_PostAbortionCaseManagement	26.1552176
BladderCancer_Investigation:HSI_BladderCancer_Investigation_Following_Blood_Urine	31.7144786
AntenatalCare_Inpatient:HSI_CareOfWomenDuringPregnancy_PresentsForInductionOfLabour	14.780422
PostnatalCare_TreatmentForObstetricFistula:HSI_PostnatalSupervisor_TreatmentForObstetricFistula	43.5566964
BreastCancer_Investigation:HSI_BreastCancer_Investigation_Following_breast_lump_discernible	3.27107271
Malaria_Treatment_Complicated_Adult:HSI_Malaria_complicated_treatment_adult	53.1153333
Malaria_Treatment_Complicated_Child:HSI_Malaria_complicated_treatment_child	52.9426498
Rti_TetanusVaccine:HSI_RTI_Tetanus_Vaccine	14.7491815
BladderCancer_Investigation:HSI_BladderCancer_Investigation_Following_pelvic_pain	31.2932924
ProstateCancer_Investigation:HSI_ProstateCancer_Investigation_Following_Pelvic_Pain	26.3227094
BladderCancer_Treatment:HSI_BladderCancer_StartTreatment	2.08523748
OtherAdultCancer_Treatment:HSI_OtherAdultCancer_PostTreatmentCheck	0.05656147
Tb_Prevention_Ipt:HSI_Tb_Start_or_Continue_Ipt	27.0567384
BreastCancer_Treatment:HSI_BreastCancer_StartTreatment	1.54286724
OtherAdultCancer_PalliativeCare:HSI_OtherAdultCancer_PalliativeCare	5.28465879
Rti_ShockTreatment:HSI_RTI_Shock_Treatment	16.0592538
Rti_BurnManagement:HSI_RTI_Burn_Management	25.1229706
Schisto_MDA:HSI_Schisto_MDA	0.71848929
AntenatalCare_FollowUp:HSI_CareOfWomenDuringPregnancy_AntenatalOutpatientManagementOfGestationalDiabetes	9.8985481
AntenatalCare_Outpatient:HSI_CareOfWomenDuringPregnancy_SixthAntenatalCareContact	12.8206602
ProstateCancer_Treatment:HSI_ProstateCancer_PostTreatmentCheck	0.02526558
BladderCancer_PalliativeCare:HSI_BladderCancer_PalliativeCare	5.55724467
ProstateCancer_PalliativeCare:HSI_ProstateCancer_PalliativeCare	5.849518
BreastCancer_PalliativeCare:HSI_BreastCancer_PalliativeCare	5.07449202
OtherAdultCancer_Treatment:HSI_OtherAdultCancer_StartTreatment	1.87277521
Epi_Adolescent_Hpv:HSI_HpvVaccine	17.0383782
Hiv_Prevention_Prep:HSI_Hiv_StartOrContinueOnPrep	3.78063555
Epi_Childhood_Bcg:HSI_BcgVaccine	21.3725594
Epi_Childhood_Opv:HSI_opv	20.6480328
Epi_Childhood_Pneumo:HSI_PneumoVaccine	20.3446482
Epi_Childhood_Rota:HSI_RotaVaccine	20.1326996
Epi_Childhood_DtpHibHep:HSI_DtpHibHepVaccine	20.3446482
AntenatalCare_Outpatient:HSI_CareOfWomenDuringPregnancy_SeventhAntenatalCareContact	1.19595856
Epi_Childhood_MeaslesRubella:HSI_MeaslesRubellaVaccine	13.4589255

What is the Effect of Using Modes 0, 1 or 2?

I examined this in the branch PR’d at #929 in src/scripts/healthsystem/impact_of_mode/analysis_health_impact_of_mode.py.

We get this result:

This is showing that in Mode 2 there is almost no benefit of the healthcare system.

This is consistent with the finding about the squeeze-factors: in Mode 2, any HSI that has a squeeze_factor > 0.0 does not run.

This is confirmed by looking at the logged HSI that run in Mode 2 (in src/scripts/healthsystem/impact_of_mode/analysis_hsi_by_mode.py):

Counts of HSI Events that Occur for each TREATMENT_ID	TREATMENT_ID	No Healthcare System	Mode 0	Mode 1	Mode 2
Alri_Pneumonia_Treatment_Inpatient	0.0	17573.2	17554.2	0.0
Alri_Pneumonia_Treatment_Inpatient_Followup	0.0	15604.8	15520.0	0.0
Alri_Pneumonia_Treatment_Outpatient	0.0	58853.6	58587.6	23.6
AntenatalCare_FollowUp	0.0	2122.8	1997.6	0.0
AntenatalCare_Inpatient	0.0	25700.6	24350.0	0.0
AntenatalCare_Outpatient	0.0	120435.0	236232.0	0.0
AntenatalCare_PostAbortion	0.0	751.4	798.0	0.0
AntenatalCare_PostEctopicPregnancy	0.0	179.8	174.8	0.0
BladderCancer_Investigation	0.0	2023.8	2314.4	0.0
BladderCancer_PalliativeCare	0.0	393.4	348.6	0.0
BladderCancer_Treatment	0.0	67.6	68.4	0.0
BreastCancer_Investigation	0.0	1479.2	1465.6	0.0
BreastCancer_PalliativeCare	0.0	112.4	121.0	0.0
BreastCancer_Treatment	0.0	625.8	592.6	0.0
CardioMetabolicDisorders_Investigation	0.0	455270.8	454498.8	0.0
CardioMetabolicDisorders_Prevention_CommunityTestingForHypertension	0.0	5061.2	4991.8	0.0
CardioMetabolicDisorders_Prevention_WeightLoss	0.0	28613.4	27167.4	0.0
CardioMetabolicDisorders_Treatment	0.0	121619.0	120402.0	0.0
Contraception_Routine	0.0	1414058.4	1430927.8	132080.2
DeliveryCare_Basic	0.0	28092.6	28341.4	0.0
DeliveryCare_Comprehensive	0.0	1963.6	1979.8	0.0
DeliveryCare_Neonatal	0.0	28898.4	29123.4	0.0
Depression_TalkingTherapy	0.0	220346.6	31942.4	0.0
Depression_Treatment	0.0	88975.8	12987.2	0.0
Diarrhoea_Treatment_Inpatient	0.0	13486.0	13416.8	0.0
Diarrhoea_Treatment_Outpatient	0.0	44370.2	44390.4	0.0
Epi_Adolescent_Hpv	0.0	3400.0	3476.6	0.0
Epi_Pregnancy_Td	0.0	46043.2	43976.4	0.0
Epilepsy_Treatment_Followup	0.0	300137.0	293249.2	0.0
Epilepsy_Treatment_Start	0.0	58824.0	57373.8	0.0
FirstAttendance_Emergency	0.0	120529.6	119730.2	82.2
FirstAttendance_NonEmergency	0.0	2265714.0	2263252.6	436.8
FirstAttendance_SpuriousEmergencyCare	0.0	14957.0	14977.2	0.0
Hiv_Prevention_Circumcision	0.0	6601.0	6545.0	0.0
Hiv_Prevention_Infant	0.0	18185.0	17800.6	0.0
Hiv_Prevention_Prep	0.0	4.75	6.2	0.0
Hiv_Test	0.0	211643.6	210123.0	8.0
Hiv_Treatment	0.0	180930.0	177754.2	0.0
Inpatient_Care	0.0	9857.6	9857.4	0.0
Malaria_Prevention_Iptp	0.0	88421.8	82702.8	0.0
Malaria_Test	0.0	4310443.4	4308986.2	0.0
Malaria_Treatment_Complicated_Adult	0.0	396.6	414.4	0.0
Malaria_Treatment_Complicated_Child	0.0	1061.6	1139.8	0.0
Malaria_Treatment_NotComplicated_Adult	0.0	38737.4	38740.0	0.0
Malaria_Treatment_NotComplicated_Child	0.0	87702.4	87650.2	0.0
Measles_Treatment	0.0	14973.0	15005.6	0.0
OesophagealCancer_Investigation	0.0	3735.0	3502.0	0.0
OesophagealCancer_PalliativeCare	0.0	72.2	59.2	0.0
OesophagealCancer_Treatment	0.0	140.0	128.6	0.0
OtherAdultCancer_Investigation	0.0	1834.0	1792.0	0.0
OtherAdultCancer_PalliativeCare	0.0	914.4	847.2	0.0
OtherAdultCancer_Treatment	0.0	1197.4	1202.0	0.0
PostnatalCare_Maternal	0.0	18757.4	18937.8	0.0
PostnatalCare_Maternal_Inpatient	0.0	2518.6	2518.2	0.0
PostnatalCare_Neonatal	0.0	21701.0	21886.6	0.0
PostnatalCare_Neonatal_Inpatient	0.0	21701.0	21886.6	0.0
PostnatalCare_TreatmentForObstetricFistula	0.0	191.8	183.2	0.0
ProstateCancer_Investigation	0.0	1577.6	1717.6	0.0
ProstateCancer_PalliativeCare	0.0	70.6	75.0	0.0
ProstateCancer_Treatment	0.0	96.2	92.4	0.0
Rti_AcutePainManagement	0.0	29978.2	30897.6	0.0
Rti_BurnManagement	0.0	3020.4	3239.2	0.0
Rti_FractureCast	0.0	248276.6	262418.6	0.0
Rti_Imaging	0.0	2883.4	2880.0	0.0
Rti_MajorSurgeries	0.0	355906.2	359030.8	0.0
Rti_MedicalIntervention	0.0	4162.2	4216.4	0.0
Rti_MinorSurgeries	0.0	447837.0	474969.4	0.0
Rti_OpenFractureTreatment	0.0	17841.8	18003.6	0.0
Rti_ShockTreatment	0.0	635.0	635.2	0.0
Rti_Suture	0.0	26802.8	26933.8	0.0
Rti_TetanusVaccine	0.0	1409.4	1418.4	0.0
Schisto_MDA	0.0	128.0	128.0	0.0
Schisto_Treatment	0.0	7239.2	6973.2	0.0
Tb_Prevention_Ipt	0.0	5171.6	5030.0	0.0
Tb_Test_FollowUp	0.0	11315.2	11403.0	0.0
Tb_Test_Screening	0.0	536404.8	533190.4	2.0
Tb_Test_Xray	0.0	2598.0	2593.6	0.0
Tb_Treatment	0.0	3463.8	3446.2	0.0
Undernutrition_Feeding	0.0	9391.6	9231.2	0.0

Observations 1) It is curious that the only HSI that occur under Mode 2 are: [‘Alri_Pneumonia_Treatment_Outpatient', 'Contraception_Routine', 'FirstAttendance_Emergency', 'FirstAttendance_NonEmergency', 'Hiv_Test', 'Tb_Test_Screening'] a. Why is this? Is this due to the lack of randomisation in the HSI queue or the structure of prioritisation in the queue? (both fixed in PR #881)? b. Should we do anything to change this? c. When we do the scale-run should be use ignore_priority=True, or otherwise change the declared priorities of HIS Events in the code (as the decisions about priorities seems a bit spurious to me).

Actions

It seems to me that the behaviour of Mode 2 is not exactly what we want because priority plays no part in determining the squeeze-factors. I think we want a new implementation whereby:

• Events are run in order of priority
• There is no squeeze-factor for any event that occur before the capacity of any HCW is exceeded.
• Squeeze-factors apply only to those HSI that occur when the capacity of a HCW has been exceeded.
• We could also include a tolerance factor that says that Mode 2 allows event runs with a squeeze_factor < tolerance; and, we set tolerance=0.0.

I would suggest creating a test for this specific behaviour using a DummyModules as follows:

• Create a Dummy Module that schedules thousands of different HSI events each day with various priorities: e.g., HSI_Dummy_Priority0, HSI_Dummy_Priority1, ….
• Run the simulation and inspect the detailed HSI_Event log in the healthsytem (retrieved through using parse_log with level=DEBUG`).
• We want to see:
  • HSI with lower priority having higher chance of squeeze_factor > 0.0 and a higher average squeeze.
  • This effect to be ameliorated if there is a greater capacity of the healthsystem (by changing the module kwarg capabilities_coefficient).
  • The tolerance parameter working as expected
  • No squeeze on the InpatientCare appointment

I would suggest that our plan is:

1. Finish PR #881
2. Branch off #881 and merge-in #929 in order to re-run the experiment about effect of model.
3. Develop the test in test_healthsystem.py (it will fail)
4. Make the necessary modification to healthsystem.py until the test passes.

[BinglingICL]

We note that almost all HSI have an average squeeze_factor > 0.0

So, in mode 1, I suppose this means for each of these HSIs, the call/availability (of HCW time) > 1, i.e., the call exceeds the availability? (And in the way that tlo delivers healthcare services, every call is met?)

[BinglingICL]

Thanks very much Tim. These are very interesting and helpful results. Looking forward to the meeting!

[tbhallett]

Thanks @marghe-molaro, @BinglingICL and @tdm32

We discussed a few things just now:

• @marghe-molaro will take a go at creating the "sandbox" environment to test the behaviour under various ways of computing 'squeeze factors' and the modes. She also proposes that we unify the concepts of tolerance, squeeze-factor and priority such that priority is inversely proportional to the maximum squeeze-factor under which that HSI will run (under Mode 2).

• @BinglingICL will look at the types of appointment that will have trouble running under the actual scenario.

• @marghe-molaro, as per Bingling's comment, please could you also check that the logic hold that, if an appointment is not possible at all (because the cadre HCW does not exist at that facility in that district) that this does induce a squeeze_factor of np.inf and that this appoinmtment does not run (in Mode 1 or Mode 2)...... if you all agree with that!?

[BinglingICL]

• @BinglingICL will look at the types of appointment that will have trouble running under the actual scenario.

Hi all, please find the appts that cannot run under mode 1/mode 2 and 'actual' HR capabilities via the link: https://docs.google.com/spreadsheets/d/1jd40eIR0Jc_rrSkPKo03M573dzF_2aTAKjouCqn9pcA/edit?usp=sharing. Would be happy to explain it in our meeting tomorrow.