HopkinsIDD / cholera-mapping-pipeline

Formerly part of cholera-taxonomy. The map creation scripts, packages, and file structure
1 stars 3 forks source link

ETH 2016-20 #398

Open eclee25 opened 1 year ago

eclee25 commented 1 year ago

Maya updated OC 21371

QLLZ commented 1 year ago

Data pull: HASH 02faaed1a21116cebc29533c68ec0d99fe78d08f

QLLZ commented 1 year ago

Data pull failed: memory issue: log file

QLLZ commented 1 year ago

Rerunning data pull with 64 G on dev: HASH: 02faaed1a21116cebc29533c68ec0d99fe78d08f

QLLZ commented 1 year ago

Data pull failed: memory issue: log file

QLLZ commented 1 year ago

Rerunning data pull with 80 G on dev: HASH: 02faaed1a21116cebc29533c68ec0d99fe78d08f

QLLZ commented 1 year ago

Model run: HASH 02faaed1a21116cebc29533c68ec0d99fe78d08f

QLLZ commented 1 year ago

Country data report

javierps commented 1 year ago

Double check over-estimation in 2016. Rest is OK.

Suggestion: temporary accept.

eclee25 commented 1 year ago

Agree with Javier --

Review data comparison report Temp Accept

javierps commented 1 year ago

2016 yearly data is from low-quality source OC 21371. OK to validate.

QLLZ commented 1 year ago

Config

Data pull (128G):

HASH: 211913e0eb7d570ed62c48bc1ae9ecd6671a6cbc

QLLZ commented 1 year ago

Model run:

HASH: 211913e0eb7d570ed62c48bc1ae9ecd6671a6cbc

QLLZ commented 1 year ago

failed model run log file

QLLZ commented 1 year ago

Rerun on dev_u_combs_fix

HASH: e56580fa5ddab00293e31ff90139351f80ae3f6c

QLLZ commented 1 year ago

country data report

QLLZ commented 1 year ago

Many years are overestiamted except for 2017 and 2018. Is this because of the large number of cases in 2017 that impac the modeled cases in other years?

2017 data from OC 20687 looks suspcious.

eclee25 commented 1 year ago

Model diags look good but some of the fits are weird and data needs to be audited. 2017 issues: Audit OC 21371, 20687

Agree with Qulu -- 2016, 2019, and 2020 data is oddly over-estimated. Would like to see an updated annual data comparison figure of this (comparing total number of cases by admin level after data filtering...)

QLLZ commented 1 year ago

OC 21371: the data is from a research paper. After double checking, the numbers look correct in our database.

OC 20687: updated observations. 405 should be weekly cases not annual cases.

javierps commented 1 year ago

Given the good convergence, I suspect this is because of different spatial patterns in case occurrence between years based on the GAM input figures, which then have the model over-estimate national level observations.

A solution may be to change the national-level od parameter to something smaller to force it to better match national-level data.

eclee25 commented 1 year ago

Rerun with updated national od parameter

QLLZ commented 1 year ago

Data pull and model run: HASH: 5c0213fae8693e4b4b84d88f0fa406548414b48d

Updated report

QLLZ commented 1 year ago

The new setting seems not working for the ETH. Convergency failed completely.

eclee25 commented 1 year ago

ID-55 diagnostics look good for ETH 2016-20 -- this had drop_censored_adm0_thresh = 1 and all other standard settings (after merging the od admin 0 parameter switch -- any country-year with >10k cases has an od_param_0 of 1000 into dev) Approve

eclee25 commented 1 year ago

Standard settings with drop_censored_adm0_thresh: 1

eclee25 commented 1 year ago

With the implementation of the new dummy censoring variables, this model can run initialize with the standard settings.

javierps commented 11 months ago

Sep 2023 production run: overall all OK, over-estimation in 2016.

Suggestion: 2nd opinion on 2016 estimates.

eclee25 commented 11 months ago

The 2016 estimates should be informed by detailed line list and sit rep data at lower admin levels and the OC 21371 source that is overestimated is a less reliable literature source. Unfortunately there is no alternate 2016 national observation because the line list is missing data from the Somali region and we only have weekly sitrep dat for Somali to supplement (so we could not run linelist2phantom). I think this is ok.

Approve