Open timriffe opened 1 year ago
Further testing adds this nuance: the place to apply something like maybe_dither_lx()
is not inside ineq_quantile_lower()
, but inside ineq_quantile()
, where we pad the top with 0 by default, ergo:
ineq_quantile <- function(age, lx, quantile = .5){
stopifnot(length(age) == length(lx))
n <- length(age)
lx <- c(lx, 0)
# suggested new line
lx <- maybe_dither_lx(lx,.0001)
a <- c(age, max(age) + 1)
qs <- rep(0,n)
for (i in 1:n){
qs[i] <- ineq_quantile_lower(age = a[i:(n+1)],
lx = lx[i:(n+1)],
quantile = quantile)
}
qs - age
}
We would also need to add it into ineq_iqr()
and ineq_cp()
, in general this is always something to do on the top level rather than in the innermost function ineq_quantile_lower()
Ties in
lx
cause problems for our quantile functions, all of which use monotonic splines. ExampleIn other applications that would come up using HMD
lx
values, warnings will also ensue.I looked into how other quantile-stuff implementations handle these situations. They use dithering. Here's an
lx
-specific dither routine that I propose, based on a subset of code found inquantreg::dither()
, given that forlx
we need to insist on monotonicity. In essence, IFF there are ties, then we generate random uniform deviates between 0 and 10000th of the radix, sort them, add them to lx, then ensure lx has the original radix.In case there are no ties, then the same lx is spit back, otherwise we do this little perturbation to eliminate ties without compromising the shape. For the above case, we get a reasonable result and no warning:
In this case, for all but the lowest ages, residuals are negative, observe:
So, trivially negative until the tail. One can simply set a default to a lower value:
This reduces most age residuals, but then squeezes larger errors to a spike at the "end". No matter what
value
we choose, warnings and errors are eliminated, but we'd of course like to reduce the consequence. Does something like this seem reasonable to you, like to just do it automatically to remove user headaches? That final "residual" is in the second case is a mere 17.68 weeks of age, or .345 years. Any thoughts on a solution in this direction?