ISARICDataPlatform / ISARIC-COVID-reports

MIT License
6 stars 2 forks source link

error on generate.report fitdist with admit.niv.2 fails #5

Closed sempwn closed 4 years ago

sempwn commented 4 years ago

I receive the following error when running generate.report after generating imported.data from import.and.process.data

<simpleError in optim(par = vstart, fn = fnobj, fix.arg = fix.arg, obs = data,     gr = gradient, ddistnam = ddistname, hessian = TRUE, method = meth,     lower = lower, upper = upper, ...): non-finite value supplied by optim>

Error in fitdist(admit.niv.2, dist = "gamma", method = "mle") : 
the function mle failed to estimate the parameters, 
with the error code 100                                                      

After a bit of investigating it seems this is happening on line 2632 of plot_function.R and is most likely due to the NIMV.start.date having many missing values. See code snippet below:

imported.data$detailed.data %>% summarise(missing = mean(is.na(NIMV.start.date)))
# A tibble: 1 x 1
  missing
    <dbl>
1   0.995

Elsewhere in the report generation there is a step that checks for missingness of the variable before running fitdist but this appears to be missing from the adm.to.niv function.

emmanuelle-dankwa commented 4 years ago

Hello, @sempwn, thank you. I've just checked this. There are checks for all the time to event functions; the issue was probably due to the fact that the criteria was not strict enough. As you rightly identified, fitdist fails for small samples.

This has now been modified: the gamma estimates will only be calculated for a variable if the number of its non-NA values is greater than 10% of the original set. Could you please check that this is now working for your site data?

sempwn commented 4 years ago

Hi @Emmanuelle7 . That's working for me now, thanks for so quickly checking and resolving the issue.