search

timriffe / DemoTools

Tools for the evaluation, adjustment, and standardization of demographic data
https://timriffe.github.io/DemoTools
Other
59 stars 31 forks source link

LTabr: transitivity #50

Open mpascariu opened 6 years ago

mpascariu commented 6 years ago

I testing that given a specified level of mortality I get the same life table no matter what indicator I use to build that life table.

In the example below I will jump from one type of input to another and check that we do not lose accuracy.

> # Sun Sep  9 00:19:45 2018 --------- Marius D. Pascariu ---
> remove(list = ls())
> library(DemoTools)
> 
> # trial code from PAS LTPOPDTH, North, Males, IMR = .1
> Exposures <- c(100958,466275,624134,559559,446736,370653,301862,249409,
+                247473,223014,172260,149338,127242,105715,79614,53660,
+                31021,16805,8000,4000,2000,1000)
> Deaths <- c(8674,1592,618,411,755,1098,1100,1357, 1335,3257,2200,4023,
+             2167,4578,2956,4212, 2887,2351,1500,900,500,300)
> 
> # Input parameters
> region <- "n" # n, e, s, w
> Sex    <- "b"    # m, f, b
> IMR    <- 0.1
> axm    <- "pas"
> Age    <- c(0, 1, seq(5, 100, by = 5))
> AgeInt <- c(diff(Age), NA)
> 
> # Build the reference LT
> LT <- LTabr(Deaths = Deaths, Exposures = Exposures, 
+             Age = Age, AgeInt = AgeInt, axmethod = axm, 
+             IMR = IMR, region = region, Sex = Sex)
> 
> # Build other 3 LTs that in principle should be identical with LT
> LT1 <- LTabr(nMx = LT$nMx, 
+              Age = Age, AgeInt = AgeInt, axmethod = axm, 
+              IMR = IMR, region = region, Sex = Sex)
> LT2 <- LTabr(nqx = LT$nqx, 
+              Age = Age, AgeInt = AgeInt, axmethod = axm, 
+              IMR = IMR, region = region, Sex = Sex)
> LT3 <- LTabr(lx = LT$lx, 
+              Age = Age, AgeInt = AgeInt, axmethod = axm, 
+              IMR = IMR, region = region, Sex = Sex)
> 
> # Test the transitive property of the LT indicators
> head(round(LT - LT1, 5), 2)  # good
  Age AgeInt nMx nAx nqx lx ndx nLx Tx ex
0   0      0   0   0   0  0   0   0  0  0
1   0      0   0   0   0  0   0   0  0  0
> head(round(LT - LT2, 5), 2)  # not good
  Age AgeInt      nMx nAx nqx lx ndx nLx Tx ex
0   0      0 -0.02103   0   0  0   0   0  0  0
1   0      0  0.00000   0   0  0   0   0  0  0
> head(round(LT - LT3, 5), 2)  # not good
  Age AgeInt      nMx nAx nqx lx ndx nLx Tx ex
0   0      0 -0.02103   0   0  0   0   0  0  0
1   0      0  0.00000   0   0  0   0   0  0  0

There seems to be a problem with the nMx in the first age interval.

timriffe commented 4 years ago

Transitivity is now achieved in all ages except the open age group, except under certain circumstances. Non-transitivity in the open age group results in uniform differences in all ages of Tx, and age-varying, but small differences in ex. An up-to-date example:

# trial code from PAS LTPOPDTH, North, Males, IMR = .1

Exposures <- c(100958,466275,624134,559559,446736,370653,301862,249409,
 247473,223014,172260,149338,127242,105715,79614,53660,
 31021,16805,8000,4000,2000,1000)

Deaths <- c(8674,1592,618,411,755,1098,1100,1357, 1335,3257,2200,4023,
 2167,4578,2956,4212, 2887,2351,1500,900,500,300)

# Input parameters
region  <- "n" # n, e, s, w
Sex     <- "b"    # m, f, b
IMR     <- 0.1
axm     <- "pas"
Age     <- c(0, 1, seq(5, 100, by = 5))
AgeInt  <- c(diff(Age), NA)

         # Build the reference L 
 LT <- lt_abridged(Deaths = Deaths, Exposures = Exposures, 
             Age = Age, AgeInt = AgeInt, axmethod = axm, 
             IMR = IMR, region = region, Sex = Sex)

# Build other 4 LTs that in principle should be identical with LT

# using output nMx from original LT
LT1 <- lt_abridged(nMx = LT$nMx, 
             Age = Age, AgeInt = AgeInt, axmethod = axm, 
             IMR = IMR, region = region, Sex = Sex)

# using nqx derived from original Deaths/Exposures
LT2 <- lt_abridged(nqx = LT$nqx, 
             Age = Age, AgeInt = AgeInt, axmethod = axm, 
             IMR = IMR, region = region, Sex = Sex)

# using lx derived from original Deaths/Exposures
LT3 <- lt_abridged(lx = LT$lx, 
             Age = Age, AgeInt = AgeInt, axmethod = axm, 
             IMR = IMR, region = region, Sex = Sex)

# again using nMx that we derived from lx (or qx)
LT4 <- lt_abridged(nMx = LT3$nMx, 
             Age = Age, AgeInt = AgeInt, axmethod = axm, 
             IMR = IMR, region = region, Sex = Sex)

# 1) Test the transitive property of the LT indicators
tail(round(LT - LT1, 5), 2)  # good
    Age AgeInt nMx nAx nqx lx ndx nLx Sx Tx ex
95    0      0   0   0   0  0   0   0  0  0  0
100   0     NA   0   0   0  0   0   0  0  0  0

# 2) start with nqx derived from original Deaths/Exposures
tail(round(LT - LT2, 5), 2)  # not good
    Age AgeInt      nMx     nAx nqx lx ndx      nLx      Sx       Tx      ex
95    0      0  0.00000 0.00000   0  0   0  0.00000 0.00742 25.82324 0.03385
100   0     NA -0.05385 0.14667   0  0   0 25.82324 0.00000 25.82324 0.14667

# 3) start with lx derived from original Deaths/Exposures
tail(round(LT - LT3, 5), 2)  # not good
    Age AgeInt      nMx     nAx nqx lx ndx      nLx      Sx       Tx      ex
95    0      0  0.00000 0.00000   0  0   0  0.00000 0.00742 25.82324 0.03385
100   0     NA -0.05385 0.14667   0  0   0 25.82324 0.00000 25.82324 0.14667

# 4) nMx derived from lx derived from original Deaths/Exposures
tail(round(LT4 - LT3, 5), 2)  # not good
    Age AgeInt nMx     nAx nqx lx ndx     nLx      Sx      Tx      ex
95    0      0   0 0.00000   0  0   0 0.00000 0.00022 0.74177 0.00097
100   0     NA   0 0.00421   0  0   0 0.74177 0.00000 0.74177 0.00421

Case 1 is fine (NA is because OAG AgeInt is NA). This is due to @patrick-gerland issue-83 the original M(omega) is preserved. Now this is upheld only for cases where nMx or Deaths, Exposures are specified in inputs.

Case 2 is understood: In cases where the input was nqx or lx, q(omega) is automatically 1, which means no information is on hand to impute m(omega). In this case, we extrapolate using MortalityLaws, and impute l(omega)/T(omega) over the extrapolation range as the m(omega). This is naturally different than the original nMx, so we expect a deviation around this big.

Case 3 is identical to case 2, since lx and nqx require no approximations to translate between each other.

Case 4 is interesting: take (a modeled) lx as the input and get nMx back from it, then make a fresh lifetable using this derived nMx. In this case, due to issue-83, the original m(omega) is maintained rather than the l(omega)/T(omega) implied by extrapolation. However, the input m(omega) should be just that, and so it should be identical, but we see it is not, and this case is one I need to understand better. I would have expected LT3 and LT4 to be identical in all ages just as LT and LT1 are.

A possible solution: since we have transitivity in all ages less omega, nMx extrapolation under the same MortalityLaws parameters (and excluding m(omega)) should be identical in all such test cases, in which case l(omega)/T(omega) should be identical, and this would achieve transitivity in the full age range if upheld. However, I don't see this happening in case 4, so there is still some detail to manage. In this case, it looks like we'd need to toggle this behavior with a new logical argument force_transitivity = FALSE or similar by default, because this would override the request of issue-83.