search

DS4PS / cpp-524-spr-2020

Course shell for CPP 524 Foundations of Program Evaluation II for Spring 2020.
http://ds4ps.org/cpp-524-spr-2020/
0 stars 1 forks source link

LAB -03 #7

Open sunaynagoel opened 4 years ago

sunaynagoel commented 4 years ago

I am having hard time understanding Chapter 11. The article explains that the pre- post design can have control group but I could not get the concept related to secular trends, measurement error and testing errors for this study.

lecy commented 4 years ago

Secular trends and testing effects are both similar in the sense that they are changes we expect independent of the treatment.

Secular trends can have many causes - they can be a result of broad economic trends, natural processes of growth (weight increase, language acquisition), decline (cognitive ability due to aging), or some other broad change that occurs in society. One fun example - did your sex ed program reduce teen pregnancy rates, or did you just happen to start your program at a similar time that the TV show "16 and Pregnant" started:

https://fortune.com/2016/04/07/16-and-pregnant-decline-pregnancy/

Testing effects, on the other hand, have a very specific cause. You give your subjects a pre-test. They learn from the pre-test, and thus the second time they take the test they score higher.

In either case, the way to address the issue is to use a group that does not receive any treatment, and observe the gains they achieve over the study period independent of the intervention. Those gains will capture both secular trends and testing effects (if an instrument is used that could cause testing effects). To remove bias, we would need to account for these trends - usually with a difference-in-difference estimator. Or the comparison group might be present to simply test the assumption that we expect no gains independent of treatments (which is the main assumption that the reflexive design requires to provide an unbiased estimate of program impact).

Measurement error is unrelated to having a control or comparison group.

sunaynagoel commented 4 years ago

Thank you @lecy. This helps a lot. I must say this chapter is trickiest of all.

lecy commented 4 years ago

Just wait until next week!

sunaynagoel commented 4 years ago

@lecy the link to submit LAB-03 does not seem to be working for me.

lecy commented 4 years ago

Should be active now, thanks.