NGA 2016-20

[QLLZ]

Data pull: HASH: 6772ea9da4f43ddc3da11c494a573af2ca3dd3c8 config

[QLLZ]

Model run: HASH: 6772ea9da4f43ddc3da11c494a573af2ca3dd3c8

[QLLZ]

Error in model run: stan_output has been created, but generated_quantity file was not created successfully. I reran this before but the same error was present.

log file

[javierps]

@QLLZ is it possible this is because of memory limitations? What was the memory limit on this job?

[QLLZ]

reran with 32 G HASH 6772ea9da4f43ddc3da11c494a573af2ca3dd3c8

[QLLZ]

country data report

[eclee25]

Convergence, fits, maps, GAM input comparison, and Rhats look good

Opinion: Approve

[javierps]

Moving this to rejected because of the over-estimation in 2018. This may be due to cells not being covered by full observations. Need to check.

[eclee25]

Rerun with same config but on the updated dev branch. New feature that clips output genquant shapefiles to adm0 output shapefile was merged

[QLLZ]

Updated config

[QLLZ]

Data pull: HASH 02faaed1a21116cebc29533c68ec0d99fe78d08f

[QLLZ]

Model run: HASH 02faaed1a21116cebc29533c68ec0d99fe78d08f

[QLLZ]

Country data report needs 64 G to run.

[QLLZ]

Country data report

[javierps]

The issue in over-estimation in 2018 is still present. This may be due to the adm0 clipping failing. National level counts seem ok.

Suggestion: Temporary accept.

[eclee25]

Do we think the overestimation might be from the high eastern border cell? Perhaps we try with higher sfrac border thresh and the fixed adm0 clip

Temp Accept

[QLLZ]

Config

Data pull(32G):

HASH: 211913e0eb7d570ed62c48bc1ae9ecd6671a6cbc

[QLLZ]

Model run:

HASH: 211913e0eb7d570ed62c48bc1ae9ecd6671a6cbc

[QLLZ]

Failed model run log file

[QLLZ]

Rerun on dev_u_combs_fix

HASH: e56580fa5ddab00293e31ff90139351f80ae3f6c

[QLLZ]

country data report

[QLLZ]

There are many observations with conflicted number of cases 2017: OC332 versus OC 20810/20902 2018: maybe using a higher censoring threshold can help 2019: OC 21138 seems suspicious (under same location period, different cases. I guess we didn't change the primary to FALSE). need to double check 2020: OC 332 is much smaller than other OCs.

Convergency looks good. Population estimates look good.

[eclee25]

Model diagnostics look ok but agree with Qulu that we need to audit some of the OCs due to apparent conflicts and weird observations. The following should be audited:

2017 issues

• OC 20810, 20902, 20747, 20729, 20737, 20713, 20721, 20766, 21138 -- these case counts seem really low compared to the WHO and other sources (OC 21179, 332)

2018 issues

• OC 20985 - seems way too low
• OC 5116 optionally - the counts seem too low but this is the UNICEF source so this is in line with what we might normally see with this source

2019 issues

• OC 21138 - way too low and lots of observations -- probably a primary = FALSE issue

2020 issues

• Verify that the large discrepancy bw WHO (OC 332) and other sources (OCs 21147, 21179) is real in the source docs

Opinion: Rerun if obs get changed in the audit. Could also consider increasing the censoring threshold

[QLLZ]

2017: except for 21138, the others are from weekly bulletin on outbreaks and other emergencies report (weekly updates) OC 20810: data is consistent with the source doc OC 20902: data is consistent with the source doc OC 20747: data is consistent with the source doc OC 20729: data is consistent with the source doc OC 20737: data is consistent with the source doc OC 20713: data is consistent with the source doc OC 20721: data is consistent with the source doc OC 20766: data is consistent with the source doc OC 21138: data is consistent with the source doc

2018: OC 20985: Updated observations - it seems to be that these cases were reported in three states in nigeria not nationwide.

2019: OC 21138: some observations were actually non-primary (stratified by sex/age groups). updates those. And for other observations, there are some conflicted data in the source (between figures). but data is consistent.

[eclee25]

Rerun standard settings due to data audit

[QLLZ]

Data pull & model run: HASH: 5c0213fae8693e4b4b84d88f0fa406548414b48d

Update country data report

[QLLZ]

OC 20985 failed to be updated. need to update this and rerun.

[eclee25]

Rerun with standard model settings - no other convergence or estimation issues need to be resolved with the model.

[QLLZ]

Model run & data pull:

HASH: 5c0213fae8693e4b4b84d88f0fa406548414b48d dev

[eclee25]

All model diags look good after the data audit. Approve

[javierps]

Sep 2023 Production run: all OK.

Under-estimation of cases in 2017 and 2018 due to conflicting national level observations. Consider higher tfrac thresh?

Suggestion: accept.

[eclee25]

If we believe that the larger case count observations are more true, I think we would need to make a different choice in the data filtering process. The estimation here looks okay given the variability in reporting across OCs.

Decision: Rerun standard settings but with a censoring threshold of 1 due to conflicting annual observations with shorter tfracs but considered full (mistakenly put tfrac threshold of 1 previously)

[eclee25]

Censoring threshold = 1 run is Approved.