search

worldbank / ietoolkit

Stata commands designed for Impact Evaluations in particular, but also data work in general
https://worldbank.github.io/ietoolkit/
MIT License
216 stars 77 forks source link

New command(s) for distributional impact analysis #204

Open MRuzzante opened 5 years ago

MRuzzante commented 5 years ago

Hi guys!

As distributional analysis is becoming more common in development economics and in policy evaluation in general (see, e.g., http://documents.worldbank.org/curated/en/292901499351272899/Distributional-impact-analysis-toolkit-and-illustrations-of-impacts-beyond-the-average-treatment-effect), I have been thinking about two packages, which may be helpful for IE folks (going beyond the current iegraph):

  1. The first is very easy and it would just be kernel densities by treatment assignment (with the option of adding descriptive statistics, such as average, median, and N, or regression estimates) on the lines of https://github.com/worldbank/Stata-IE-Visual-Library/blob/master/Library/Density%20plots/Density%20plot%20with%20averages)
  2. The second would plot quantile treatment effects (see https://github.com/worldbank/Stata-IE-Visual-Library/tree/develop/Library/Regression%20coefficients/Quantile%20treatment%20effect). There are some packages out there: qreg and sqreg, the Stata ones, do not include clustered standard errors option, while qreg2 (user-written) allows clustered standard errors but does not plot the estimates.

If you agree these may be of interest, I am happy to discuss with you and eventually help draft a version of these packages.

Cheers! Matteo

luizaandrade commented 5 years ago

Thanks for the suggestion, @MRuzzante! Here are some comments:

  1. I think this is straightforward to implement and useful. We should probably start with this one, unless @kbjarkefur, @bbdaniels or @roshni13khincha have objections.
  2. My reservation with this command is that we cannot user user-written commands inside user-written commands. Maybe the easiest solution to this problem would be to ask StataCorp to include clustered standard errors to the existing commands? Otherwise, we'd have to implement everything from the estimation to the plot. It's possible that some of this is done in DIA toolkit. If you think it's easiest to just use qreg2 and create a qreg2plot command, we can maybe try to get in touch with the authors to see if they'd like to add it to their package -- or just do it as your first SSC command! I'm happy to discuss and help you with it either way.
bbdaniels commented 5 years ago

Yeah I think a great option would be to get DIA-toolkit shaped up if it does the right kind of things you are talking about. I think Kris knows a lot about it. It is more complex coding though.

jpazvd commented 5 years ago

The grqreg command plots coefficients of stata official quantile commands, such as qreg and sqreg. It think i had incorporated the support to qreg2. If not this can be easily done.

MRuzzante commented 5 years ago

Hi @jpazvd! Thanks for your input!

It seems that grqreg only works with SSC commands, suchs as qreg, bsqreg or sqreg. This means it currently does not allow to plot confidence intervals based on quantile regressions with clustered standard errors.

jpazvd commented 5 years ago

But we can add this feature in this command. It prevents the unnecessary atomization of commands. It should be trivial. Best, JP

MRuzzante commented 5 years ago

That would be great, @jpazvd!

All the best, Matteo

luizaandrade commented 5 years ago

I know I'm late for the party, but just wanted to say I think that's great! I agree we should try to aggregate commands instead of atomizing them. @jpazvd, is there a repo for this command where we can follow the implementation for qreg2? I think @MRuzzante also started doing some work on it for his paper, so if there is a repo, he may be able to share some code or make a pull request.