search

WLenhard / cNORM

Continuous Norming with R
https://www.psychometrica.de/cNorm_en.html
GNU Affero General Public License v3.0
2 stars 2 forks source link

Confidence interval in the normTable function: bug and suggestion #4

Closed hynekcigler closed 3 years ago

hynekcigler commented 3 years ago

I probably found a bug in normTable function. If confidence intervals are requested, the interval for percentile works, but it is definitely wrong for the standard score as it shows the upper bound in both columns. An example (using N(10,3)): image

In addition, I really like you use regression method. However, I would like to know more about the exact method which is used. Is it SE for observed score, SE = SD*sqrt(1-r), or SE for true-score estimate, SE_t = SD*sqrt(r)*sqrt(1-r)? And the regression was estimated like E(t) = r*X+(1-r)*M? It would be helpful to provide formula or reference any source with them.

WLenhard commented 3 years ago

Dear Hynek, many thanks for the feedback. Yes, this is clearly wrong. The bug is not in the normTable function itself, but in a 'prettyPrint' function that helps to collapse rows with equal scores and there, the lower CI bound was misspecified. I will commit the corrected code in the course of this day. Meanwhile you can avoid this by providing the parameter 'pretty = F' in the normTable function.

CI calculation: The function uses the true score estimate and afterwards corrects for the regression to the mean. I will add documentation accordingly. The formula is as follows: https://www.psychometrica.de/img/ci.png (sorry, in German, but the math should be understandable). It is based on a German text book for psychometrics: Amelang & Schmidt-Azert, 2006 (p. 52).

WLenhard commented 3 years ago

P.S.: Here is the formula from Eid & Schmidt (2012): Eid_Schmidt-2012_CI

The regression to the mean then just shifts the CI.

hynekcigler commented 3 years ago

Great, thank you, I missed the option pretty=F, which would help me a lot as I need to produce table similar to this one: image (Standard score at sides, in the table ranges of raw scores for each subtest). I am trying to prepare a function to produce such a table; if you were interested, I could provide it.

Thank you also for the formula. It is standard error for the true score estimation, while I usually prefer standard error of the observed score (constructed around the true score estimate, as suggested by Dudek, 1979, or Harvill, 1991). Some references in English, if you need them for the future update:

  1. Dudek, F. J. (1979). The Continuing Misinterpretation of the Standard Error of Measurement. Psychological Bulletin 86(2), 335-337. https://doi.org/10.1037/0033-2909.86.2.335
  2. Harvill, L. M. (1991). Standard Error of Measurement. Educational Measurement: Issues and Practice, 10(2), 33–41. https://doi.org/10.1111/j.1745-3992.1991.tb00195.x
  3. Charter, R. A., & Feldt, L. S. (2001). Confidence intervals for true scores: Is there a correct approach? Journal of Psychoeducational Assessment, 19(4), 350–364. https://doi.org/10.1177/073428290101900404

PS.: I love cNORM :-) I recommended it to my colleague two years ago, and so it has been used for Czech adaptation of Woodcock-Johnson IV. Now, I am working on TOMAL-SE. It is really great!

WLenhard commented 3 years ago

Many thanks for the suggestion and the literature! We answered via mail. Best regards!