CSBLatvia / vardpoor

Variance Estimation for Sample Surveys by the Ultimate Cluster Method
https://csblatvia.github.io/vardpoor/
European Union Public License 1.2
10 stars 8 forks source link

F'(quantile) in linrmir #4

Closed DjalmaPessoa closed 8 years ago

DjalmaPessoa commented 9 years ago

To estimate the derivative of the cumulative distribution function, Osier(2009) proposes the formula:

$$F'_K(x)= \frac{1}{\sqrt(2\pi)}.\frac{1}{Nh}\sum_{k\in U} \exp\left[-\frac{(x-x_k)^2} {2h^2} \right]$$

with h estimated by:

$$\hat{h}= \frac{\hat{\sigma}}{N^{-1/5}}$$

Considering h fixed and defining the variable z:

$$z_k = \frac{1}{h\sqrt(2\pi)} \exp\left[-\frac{(x-x_k)^2} {2h^2}\right]$$

Then, F'_{K(x)} is the mean of z:

$$F'_{K(x)}= \frac{\sum_{k\in U}z_k} {N}$$

An estimate of the mean of z in the domain C is

$$\hat{F'}_K(x)= \frac{\sum_{k_\in C}z_kw_k}{\sum_{k_\in C}w_k}= \frac{\sum_{k=1}^n I_Cz_kw_k}{\sum_{k=1}^n I_Cw_k}$$

where I_C is the indicator of the domain.

In the function linrmir, to estimate the derivative in the quantile quant_under for people aged below 65 it was used:

1. Linearization of the median income of people aged below 65

            u1 <- (quant_under_65-Y1) * dom1/h
            vect_f1 <- exp(-(u1^2)/2)/sqrt(2*\pi)
            f_quant1 <- sum(vect_f1*wght)/(N1*h)   # Estimate of F'(quantile) in linrmir

From this, outside the domain, the values of u1 are 0 and the correspondent values of vec_f1 are equal to 1/ \sqrt{2\pi}, independently of the distance from x_k to x.

The denominator N1 estimates the domain population and just adds the weights for observations in the domain while in the numerator, because exp(0)=1, all observations take part in the sum.

The estimate \hat{F'}_K(x) using the same notation would be:

2. Linearization of the median income of people aged below 65

            u1 <- (quant_under_65-Y1)/h
            vect_f1 <- exp(-(u1^2)/2)/sqrt(2*pi)
            f_quant1 <- sum(dom1*vect_f1*wght)/(N1*h)   # Alternative Estimate of F'(quantile) 

the same applies for the value of the derivative at quant_over.

I wonder if dom1 was misplaced?

References:

Guillaume Osier (2009). Variance estimation for complex indicators of poverty and inequality. Journal of the European Survey Research Association, Vol.3, No.3, pp. 167-195, ISSN 1864-3361, http://ojs.ub.uni-konstanz.de/srm/article/view/369.

pessoad@gmail.com

djhurio commented 9 years ago

I tried to clean up your question. Please amend it if I have messed up something. Please not LaTeX code is not supported here.

ajdamico commented 9 years ago

hi, djalma sent you a pdf with properly-formatted latex. github isn't letting us upload it here (i have submitted a bug report to them)

ajdamico commented 9 years ago

hi, are the recent commits related to this issue? thanks!

https://github.com/CSBLatvia/vardpoor/commits/master

djhurio commented 8 years ago

we do not have an answer yet on this. We are still working on it.

ajdamico commented 8 years ago

hi, any chance your team might be able to take a look at this? thanks!

ajdamico commented 8 years ago

hi, github now allows me to upload a zipped file.. does the CSB team think it might be able to look at this discrepancy? thanks!

derivative.zip

JBreidaks commented 8 years ago

In vardpoor version 0.7.0 is corrected mistake in linrmir and in linarr.