A question on random effect variance

[StanleyYuSu]

Hi, I was testing the example code for univariate model with non-sampled data. Below is the code I ran: data(datasaeu.ns) Fo = y ~ x1 + x2 vardir = "vardir" MSE.ns3 <- mseFH.ns.uprop(Fo, vardir, cluster = datasaeu.ns$cluster, data = datasaeu.ns) The random effect variance, which will be used to construct the shrinking factor, is given by MSE.ns3$fit$refvar [1] 0.9278286 I was wondering how this value was calculated. I thought the random effect variance is related to the Mean Square Error (MSE.ns3$mse$PC), which was estimated from the 1000 iterations (by default) of bootstrapping. But how exactly was the random effect variance of 0.9278286 derived? How can I reproduce this value? I appreciate any of your advice. Thank you and have a good day! Stanley

[mrijalussholihin]

Hi @StanleyYuSu,

Thanks for the questions.

But how exactly was the random effect variance of 0.9278286 derived?

First I need to make it clear that this package is for proportion. Proportion data has limit of 0 to 1. This is a problem to EBLUP because it assume that the data is normally distributed and has no limit (-Inf to +Inf). So, this package take the approach of transforming the proportion data (using alr a.k.a. logit) to make sure the data make sense for EBLUP. Then, the result of EBLUP is inversed to its proportion form, this is PC.

Back to the question, the random effect variance is calculated using Restricted Maximum Likelihood (REML) after the proportion data is transformed, not in its original form. If non-sampled cases are present, random effects variance is calculated only from the sampled areas, so the non-sampled areas are excluded.

For more detail about this approach, you can read it here: Model: https://doi.org/10.1007%2Fs11749-019-00688-w If non-sampled area is present: https://doi.org/10.9790%2F5728-10121519

How can I reproduce this value?

If you are curious about the algorithm, you can read the syntax of the functions. I wrote some comments in it. Hopefully it's clear enough.

Maybe it's also necessary for us as the maintainer to write a vignettes for this package to make the methods used are clear. Hopefully we can write it and update it soon. Thanks for the suggestion.

If you have more questions, please feel free to ask. Thank you and have a good day as well! Rijal

[StanleyYuSu]

Hi Rijal, Thank you very much for taking the time to answer my question. Yes, the sae.prop package is very relevant to my study, which is to estimate the proportion of businesses answering Yes (vs. No) to a specific survey question in small domains. This is the context where binomial logit model is applied.

I read your syntax (https://github.com/cran/sae.prop/blob/master/R/mseFH.ns.uprop.R) but was not quite sure I understood. Typically in Line 22: \item \code{refvar} : estimated random effects variance. How could I find the exact formula to derive refvar? Thank you again! Stanley

[mrijalussholihin]

Hi again @StanleyYuSu,

How could I find the exact formula to derive refvar?

You can find it in here, "Small area estimation of proportions under area-level compositional mixed models" by Esteban et al. (2019). It is on the Supplementary material 1, Appendix C: REML Fisher-scoring algorithm: https://link.springer.com/article/10.1007/s11749-019-00688-w#Sec8

or you can find similar approach in "Small Area Estimation 2nd Edition" by Rao and Molina (2015) on section 5.2.4. ML and REML Estimators.

Hope it helps.

======

Oh yeah, please note that estimated MSE for non-sampled areas tend to be underestimated, meaning the current method tends to give lower estimate for MSE than it should be. We are still looking for a way to solve this.

But for sampled areas, you are good to go. They give pretty reliable estimate of MSE.

Best regards, Rijal

[StanleyYuSu]

@mrijalussholihin Thank you Rijal! That's really helpful! Stanley

[mrijalussholihin]

@StanleyYuSu Glad it helps. If you have any question or problem while using sae.prop package, please let us know.

This package is still very nascent, so we still need a lot of feedback to improve this package.

Best regards, Rijal