robjhyndman / demography

demography package for R
https://pkg.robjhyndman.com/demography
73 stars 25 forks source link

Handling lots of zeros #5

Open robjhyndman opened 11 years ago

robjhyndman commented 11 years ago

When there are lots of zeros in deaths, births and population numbers, the package can fail.

Need to: make existing functions more robust to zeros. (Test on Faroe Islands data).

Could add: interpolation using a bivariate smoother on age-time mortality surface.

ArthurRose commented 10 years ago

Hallo Prof. Hyndman, I noticed that your package can not handle zeros well. It can interpolate zeros if there is at least 1 non zero entry per age group. If an entire age group is zero it will fail (LC function).

For data for Curacao I used your fill.zero functions over the age groups and subsequently over the years.

I used your fill.zero interpolation function on the rows, as intended, with the slight modification that it will skip a row containing only zero. In a second pass over the columns it applies the same function and interpolates over the columns. My data is for Curacao, a small island in the Caribbean.

You might want to add a Weighted Least Squared minimalization method with dummy substitution to find the parameters for LC regardless of the zeros and not just SVD on the log of the mortality rates.

Lifemetrics uses MLE and is not able to handle zeros either without modifying the code, so you packages can gain in popularity by adding various methods to fit lee-carter. WLS, MLE ...(Wilmoth, 1993)

Is it correct to interpolate over the rows and than the columns?

ArthurRose commented 10 years ago

Package ‘MortalitySmooth’, Author Carlo G Camarda