Issue with enforcing two-day limit on delay of care in ANC modules

[marghe-molaro]

When running the simulation in mode_appt_constraints = 2 and allowing women to persist in seeking care (such that they may receive care over two days later than topen) the following issue presents with some of the women receiving a generic_first_appt_emergency:

File "/TLOmodel/src/tlo/methods/healthsystem.py", line 2615, in apply self.process_events_mode_2(hold_over) File "/TLOmodel/src/tlo/methods/healthsystem.py", line 2432, in process_events_mode_2 actual_appt_footprint = event.run(squeeze_factor=squeeze_factor) File "/TLOmodel/src/tlo/methods/healthsystem.py", line 224, in run updated_appt_footprint = self.apply(self.target, squeeze_factor) File "/TLOmodel/src/tlo/methods/hsi_generic_first_appts.py", line 97, in apply do_at_generic_first_appt_emergency(hsi_event=self, squeeze_factor=squeeze_factor) File "/TLOmodel/src/tlo/methods/hsi_generic_first_appts.py", line 313, in do_at_generic_first_appt_emergency mni[person_id]['sought_care_for_complication'] & KeyError: 'sought_care_for_complication'

In a year-long test, this error occurred 9 times. It is most probably linked to PR #1027.

This error does not present itself if not allowing women to continue to seek care after they have been turned away (i.e. if tclose = topen + 1 day).

[marghe-molaro]

Over a 5.5 years period (and as queues get longer), this error ramps up to an average of 76 of such errors a year

[marghe-molaro]

Additional errors encountered which are related to this issue:

1. At the generic_first_appt_emergency, the person is in labour but cannot be found in the mni dictionary;

2. Following error, again related to missing keys in the mni dictionary: File "/TLOmodel/src/tlo/simulation.py", line 279, in fire_single_event event.run() File "/TLOmodel/src/tlo/events.py", line 66, in run self.apply(self.target) File "/TLOmodel/src/tlo/methods/postnatal_supervisor.py", line 924, in apply self.module.set_postnatal_complications_mothers(week=week) File "/TLOmodel/src/tlo/methods/postnatal_supervisor.py", line 717, in set_postnatal_complications_mothers self.apply_risk_of_maternal_or_neonatal_death_postnatal(mother_or_child='mother', individual_id=person) File "/TLOmodel/src/tlo/methods/postnatal_supervisor.py", line 842, in apply_risk_of_maternal_or_neonatal_death_postnatal potential_cause_of_death = pregnancy_helper_functions.check_for_risk_of_death_from_cause_maternal( File "/TLOmodel/src/tlo/methods/pregnancy_helper_functions.py", line 494, in check_for_risk_of_death_from_causematernal risk = {cause: params[f'cfr{cause}']} KeyError: 'cfr_postpartum_haemorrhage'

[marghe-molaro]

Hi @joehcollins @tbhallett,

Another issue has arisen which seems to be linked to the new "inpatient_care_doesnt_run" fnc (see error below). This occurred during a tclose = 1day run.

File "/TLOmodel/src/tlo/methods/healthsystem.py", line 2334, in process_events_mode_2 self.module.call_and_record_never_ran_hsi_event( File "/TLOmodel/src/tlo/methods/healthsystem.py", line 1780, in call_and_record_never_ran_hsi_event hsi_event.never_ran() File "/TLOmodel/src/tlo/methods/care_of_women_during_pregnancy.py", line 2380, in never_ran self.module.inpatient_care_doesnt_run(self.target) File "/TLOmodel/src/tlo/methods/care_of_women_during_pregnancy.py", line 1366, in inpatient_care_doesnt_run self.sim.modules['PregnancySupervisor'].apply_risk_of_death_from_monthly_complications(individual_id) File "/TLOmodel/src/tlo/methods/pregnancy_supervisor.py", line 1574, in apply_risk_of_death_from_monthly_complications potential_cause_of_death = pregnancy_helper_functions.check_for_risk_of_death_from_cause_maternal( File "/TLOmodel/src/tlo/methods/pregnancy_helper_functions.py", line 435, in check_for_risk_of_death_from_cause_maternal if (mother.ps_htn_disorders == 'severe_pre_eclamp' and mni[individual_id]['new_onset_spe']) or \ KeyError: 56297

[marghe-molaro]

Based on test carried out in effect_of_policy-2023-09-19T153053Z (https://github.com/UCL/TLOmodel/tree/b28816fdd3fc805156d11645143b2c13f13283ab) error appears to occur whether switching from mode 1 to mode 2, or whether sticking to mode 2 the entire time.

[marghe-molaro]

Note that "natural" tclose assumption in labour module (which leads to error) is already <= 2 days, hence why it doesn't lead to clash with functions supposed to ensure no care can take place past 2 days of delay.

[marghe-molaro]

Run effect_of_policy__2023-09-07T081534Z based on https://github.com/UCL/TLOmodel/tree/d4898df06abfa9e2d93e5116e105b0e280a918c3 shows that error is due to mother being listed in the mni dictionary, but sought_care_for_complication entry cannot be found (KeyError) in it. This is very puzzling as this doesn't seem to be removed anywhere in the code, and other keys seem to be present (i.e. dictionary is not empty).

Example when raising exception: ["This person is in mni dictionary but doesn't have sought_care_for_complication: {'delay_one_two': True, 'delay_three': False, 'delete_mni': True, 'didnt_seek_care': False, 'cons_not_avail': False, 'comp_not_avail': False, 'hcw_not_avail': False, 'ga_anc_one': 0, 'anc_ints': [], 'abortion_onset': Timestamp('2011-11-11 00:00:00'), 'abortion_haem_onset': Timestamp('2011-11-11 00:00:00'), 'abortion_sep_onset': Timestamp('2011-11-11 00:00:00'), 'eclampsia_onset': NaT, 'mild_mod_aph_onset': NaT, 'severe_aph_onset': NaT, 'chorio_onset': NaT, 'chorio_in_preg': False, 'ectopic_onset': NaT, 'ectopic_rupture_onset': NaT, 'gest_diab_onset': NaT, 'gest_diab_diagnosed_onset': NaT, 'gest_diab_resolution': NaT, 'mild_anaemia_onset': NaT, 'mild_anaemia_resolution': NaT, 'moderate_anaemia_onset': NaT, 'moderate_anaemia_resolution': NaT, 'severe_anaemia_onset': NaT, 'severe_anaemia_resolution': NaT, 'mild_anaemia_pp_onset': NaT, 'mild_anaemia_pp_resolution': NaT, 'moderate_anaemia_pp_onset': NaT, 'moderate_anaemia_pp_resolution': NaT, 'severe_anaemia_pp_onset': NaT, 'severe_anaemia_pp_resolution': NaT, 'hypertension_onset': NaT, 'hypertension_resolution': NaT, 'obstructed_labour_onset': NaT, 'sepsis_onset': NaT, 'uterine_rupture_onset': NaT, 'mild_mod_pph_onset': NaT, 'severe_pph_onset': NaT, 'secondary_pph_onset': NaT, 'vesicovaginal_fistula_onset': NaT, 'vesicovaginal_fistula_resolution': NaT, 'rectovaginal_fistula_onset': NaT, 'rectovaginal_fistula_resolution': NaT, 'date_anc_admission': NaT, 'date_preg_emergency': NaT, 'test_run': False, 'pred_syph_infect': NaT, 'new_onset_spe': False, 'cs_indication': 'none'}"]} {"uuid": "d24b453763", "date": "2011-11-12T00:00:00", "values": [1, "F", "Other", "Other", 51497, 3, false]}

[marghe-molaro]

Hi @joehcollins, Here is a summary of the current understanding of issue:

mni = sim.modules['PregnancySupervisor'].mother_and_newborn_info and labour_list = sim.modules['Labour'].women_in_labour appear to be out of sink during a do_at_generic_first_appt_emergency in one of the following two ways:

1. The person is in the labour_list but not in the mni at all;
2. The person is in both the labour_list and the mni dictionary, but the latter doesn't include any of the labour variables (although it does include all variables added during ), suggesting the person didn't experience a LabourOnsetEvent despite being listed among the women in labour. In this case, it appears some of the women had the 'delete_mni' variable set to True, and some to False.

[joehcollins]

Just noticing now on first look that self.women labour is defined twice (lines 51 & 870 - doubt this is causing any issues though). should be fixed

[joehcollins]

@marghe-molaro just jotting down my thoughts here but not sure if i'm 100% correct. Issue seems to occur when specifing these parameters to the health system: 'HealthSystem': { 'cons_availability': "default", "use_funded_or_actual_staffing": "actual", 'year_policy_switch': 2011, 'mode_appt_constraints_postSwitch': 2, "policy_name_post_switch": "VerticalProgrammes", 'tclose_overwrite': 1, 'tclose_days_offset_overwrite': 10, }

Does this sound possible: 1.) Due to these parameters tclose for labour HSIs is being over ridden and is now 10 days after topen

2.) For women that HSI_Labour_ReceivesSkilledBirthAttendanceDuringLabour never runs due to high squeeze, the HSI function self.never_ran is being called by the health system module at the updated tclose which is several days after some of the main labour events: https://github.com/UCL/TLOmodel/blob/d4898df06abfa9e2d93e5116e105b0e280a918c3/src/tlo/methods/healthsystem.py#L1304

This is dependent on if I'm following this correctly, lowest priority when debugging in the code seems to be 2?

3.) self.never_ran for HSI_Labour_ReceivesSkilledBirthAttendanceDuringLabour was then scheduling LabourAtHomeEvent to run for these women

4.) Within LabourAtHomeEvent risk of complications is being applied again to women (inappropriately), some of whom are seeking care via HSI_GenericNonEmergencyFirstAppt

5.) HSI_GenericNonEmergencyFirstAppt may run immediately OR may also now be delayed up to this new tclose by which point women are no longer in the labour list OR may have died leading to removal of their entry from the MNI

This late running of the never_ran function is supported by the logging (i set everything from the labour modules to default). I was also seeing multiple error messages from my modules related to event scheduling. I have added some fixes on a branch i'm working on which has resolved all the error messages I saw previously.

I have not confirmed if the original error messaging is fixed because ive removed scheduling of emergency care seeking during a home birth occuring through HSI_GenericNonEmergencyFirstAppt as it seemed very complicated with all the possible delays and variable topen.

Does this sound about right - i dont think i will have this fixed by tomorrow but hopefully we know the route cause now.

[joehcollins]

I'm just running your scenario for 5 years and will have a look to see if any error codes are persisting

[joehcollins]

Also just looking above I need to think about the errors being generated from ANC also