search

danielwilhelm / STATA-ME-test

This project provides STATA commands for testing for the presence of measurement error
MIT License
1 stars 1 forks source link

Your Stata paper #1

Open ericmelse opened 2 years ago

ericmelse commented 2 years ago

Dear Professor Wilhelm,

With interest I am exploring your Stata package dgmtest.

Maybe you should include your own paper published in the Stata Journal in the References section of your Github page: Lee, Y. J., & Wilhelm, D. (2020). Testing for the presence of measurement error in Stata. The Stata Journal, 20(2), 382–404. https://doi.org/10.1177/1536867X20931002

Regarding dgmtest, would it be possible to apply weights (like [pw], [aw])? This is of importance with surveys that are not representative for a particular population but with weights are. My concern is how dgmtest would function in either case, or, could using dgmtest with the data without weights result in signalling measurement error whereas - if weights could be applied - the result might be that measurement error is much less an issue.

Time permitting I appreciate your comment on this subject.

ericmelse commented 2 years ago

Dear Professor Wilhelm,

In the Stata Journal dgmtest installation page: package st0600 from http://www.stata-journal.com/software/sj20-2 the following line is written:

emp.do requires a dataset that must be installed from ICPSR.

I tried to locate the record for this dta file in https://www.openicpsr.org/ but so far I was not able to find the source of your dta file: "cps-cleaned.dta" Could you provide a link to the web page where it can be downloaded or possibly add the dta file to your Github page of dgmtest?

danielwilhelm commented 2 years ago

Dear Professor Wilhelm,

With interest I am exploring your Stata package dgmtest.

Maybe you should include your own paper published in the Stata Journal in the References section of your Github page: Lee, Y. J., & Wilhelm, D. (2020). Testing for the presence of measurement error in Stata. The Stata Journal, 20(2), 382–404. https://doi.org/10.1177/1536867X20931002

Regarding dgmtest, would it be possible to apply weights (like [pw], [aw])? This is of importance with surveys that are not representative for a particular population but with weights are. My concern is how dgmtest would function in either case, or, could using dgmtest with the data without weights result in signalling measurement error whereas - if weights could be applied - the result might be that measurement error is much less an issue.

Time permitting I appreciate your comment on this subject.

Dear Eric Melse,

thank you for your interest in our Stata package.

  1. The Stata command currently does not support weights, but I agree that would be useful; we'll add it to the todo list. Asymptotically, i.e. in large enough samples, assuming that sampling weights are nonzero, the test has power against all alternatives regardless of whether you reweighs or not. Also, both versions of the test control size asymptotically. Therefore, asymptotically, the problem of signalling measurement error with weights while not finding measurement error without weights cannot occur.

In finite (i.e. small) samples, however, it is possible that sampling weights improve power of the test, so that the weighted version rejects while the unweighted version does not. Actually, the opposite can occur, too: the weighted test may have less power than the unweighted one. In small samples, which test is more powerful depends on the data-generating process.

I hope this helps. I suppose my conclusion is: if you have a reasonably large sample, you can apply the currently implemented (unweighted) version of the test.

  1. I do not have permission to include the data in the package. This is the CPS-SER dataset that is needed: https://www.icpsr.umich.edu/web/ICPSR/studies/9039/datadocumentation#. Once downloaded, the data is cleaned and the sample selected as described in the paper. I'm attaching my do file that performs the steps to create cps-cleaned.dta (I renamed it as .txt file because GitHub doesn't support .do files in these conversations).

Best, Daniel generatedataset.txt