Plot several models from xtevent in one plot with xteventplot

[izmartinez]

It would be useful if one could plot several event study models into one plot using xteventplot.

Currently, one can in principle resort to the user-written command coefplot. However, for this it would be useful if there was an estimate _k_eq_m1 saved for t-1 (and similarly, all the other pre-treatment time points when adding the trend(#) option to the estimation) in the coefficient vector e(b).

[jorpppp]

@rayhuang11 will look into this. The first step is to produce a event study plot with two coefficient paths next to each other, from two coefficient and variance matrices. Then, we can look into doing the same from xtevent output.

Also look at the other DID commands out there, you can find them in the Asjad Naqvi guide. At least one of those commands has syntax for a multiple event study plot that we may be able to use.

[rayhuang11]

Update: did not do any work on this in the past week.

[rayhuang11]

@Constantino-Carreto-Romero @jorpppp I may have found a potential bug. When I run

xtevent y eta , panelvar(i) timevar(t) policyvar(z) window(5) plot

on the example.dta and try to get the x-axis values for the plot via e(mattrendx), I get the error r(111) matrix e(mattrendx) not found. The same applies to e(mattrendy).

[rayhuang11]

The first step is to produce a event study plot with two coefficient paths next to each other, from two coefficient and variance matrices. Then, we can look into doing the same from xtevent output.

Update: successfully created some fake data produced an event study plot with two coefficients paths next to each other from the matrices taken from the xtevent command.

For future reference: the link for of Asjad Naqvi's guide on DiD is here.

[jorpppp]

@Constantino-Carreto-Romero @jorpppp I may have found a potential bug. When I run

xtevent y eta , panelvar(i) timevar(t) policyvar(z) window(5) plot

on the example.dta and try to get the x-axis values for the plot via e(mattrendx), I get the error r(111) matrix e(mattrendx) not found. The same applies to e(mattrendy).

This is not a bug, just lack of clarity in the documentation. We'll edit the documentation in #71.

[jorpppp]

We are thinking that to make this work, we need to create a command that makes repeated calls to xteventplot to generate the x and ya values to be plotted from multiple model estimates. The first step to implement this is to add an offset option to xteventplot that offsets the x-values of the event coefficients by a fixed amount.

@rayhuang11 will check Asjad Naqvi's guide to see if there's another easier way to do this. If not, he'll start by adding the offset option to xteventplot.

[rayhuang11]

@rayhuang11 will check Asjad Naqvi's guide to see if there's another easier way to do this.

@jorpppp I did not find anything in Naqvi's guide.

[jorpppp]

Thanks @rayhuang11, we'll implement the offset option then.

[jorpppp]

@rayhuang11 will also check the eventdd package

[rayhuang11]

No updates from @rayhuang11 for this week.

[jorpppp]

Per this comment, there's a function in the did_imputation package that we may want to look at:

https://github.com/JMSLab/xtevent/issues/35#issuecomment-1206498863

[rayhuang11]

Per this comment, there's a function in the did_imputation package that we may want to look at: https://github.com/JMSLab/xtevent/issues/35#issuecomment-1206498863

The did_imputation package uses event_plot.ado to plot up to 8 models in the same graph. Here is the example from their repo:

The offset here is using the default distance which is not user specified. Rather, the distance depends on how many models are being plotted together. I've imitated a version of their offset command in https://github.com/JMSLab/xtevent/commit/386fd82c4aedf3e6b458b8f01b9737345ffeead3 to the issue7_RH_test.do file. @jorpppp do we prefer this version or the version where the offset is manually specified by the user as we initially discussed (e.g. offset(0.2) would offset each model by 0.2)?

[rayhuang11]

Current approach:

Taking inspiration from event_plot.ado from the did_imputation package, we can adapt xteventplot so that when we use xteventplot the way we have been doing so all along, everything stays the same (i.e. calling xtevent then xteventplot.

However, if the user specifies xteventplot model1 model2 model3..., options xteventplot will graph the several models together. The user can access the saved results from the different models through the estimates store command. Then xteventplot will run the existing code in a loop for each model, then take the stored results and overall them on top of one another. The main code we would be adding would be an offset option.

My main concerns at this point would be how all the options would interact with this approach. E.g. if we used the option noci, we would force all the models to not display confidence intervals.

Note: we would have to pick a max number of models to allow. Currently, did_imputation allows for up to 8.

I also uploaded event_plot.ado here.

Fyi @jorpppp @Constantino-Carreto-Romero.

[rayhuang11]

@Constantino-Carreto-Romero thanks for the help yesterday!

For today's meeting, we should go over my plan to adapt xteventplot.ado to allow for the plotting of multiple models in https://github.com/JMSLab/xtevent/commit/762be427653d800e34ccfd88d4ddd443d7a89d3c.

What options will be incompatible with multiple models? e.g. I don't think we should allow p-values to be displayed.

[izmartinez]

Thank you all for your work and service to the research community!

I'm working on a new project where we'll use event studies, too, and I'm already looking forward to these new features you're adding.

[jorpppp]

@Constantino-Carreto-Romero thanks for the help yesterday!

For today's meeting, we should go over my plan to adapt xteventplot.ado to allow for the plotting of multiple models in 762be42.

What options will be incompatible with multiple models? e.g. I don't think we should allow p-values to be displayed.

@rayhuang11 I think the pseudocode here is the proper way to go. Some thoughts about how to handle xteventplot options per call today:

• Options noci, nosupt, scatterplotopts, suptplotopts, ciplotopts, and suptreps should work as repeated options, as in the following syntax: xteventplot model1 model2, ci(ci noci) supt(supt nosupt) scatterplotopts(mcolor(red green))
• Options nozeroline should apply to all the plotted models.
• Labels for the value at minus 1 should be disabled..
• p-values should be disabled
• Overlays should be disabled
• y and proxy should be disabled (for now, there may be a way to implement them but it's unnecessary now)
• Levels should apply to alll the models.
• smpath should be disabled (for now, this is an interesting problem to think about in our next call with @SimonFreyaldenhoven and @chansen776 )
• overid, overidpost disabled (for now)
• smplotopts, staticovplotopts, trendplotopts, addplotopts, textboxopts should be disabled

[rayhuang11]

Status update

Thanks @Constantino-Carreto-Romero for the help!

As of https://github.com/JMSLab/xtevent/commit/20fc0374af3643a4fa17b62c2f6ed87b7d692e89, xteventplot can now plot multiple models with the offset. Currently, we require that the normalized coefficient be the same for all the models.

All the disabled options requested in https://github.com/JMSLab/xtevent/issues/7#issuecomment-1213340711 have been disabled. Options that should apply to all models are work as intended.

• Options noci, nosupt, scatterplotopts, suptplotopts, ciplotopts, and suptreps should work as repeated options, as in the following syntax: xteventplot model1 model2, ci(ci noci) supt(supt nosupt) scatterplotopts(mcolor(red green))

This part is almost working as intended. There is a small bug that I have yet to address involving tokenize and local positional macros. Namely, I call tokenize multiple times, and for the second time I call tokenize, the local macros in memory still refer to the first tokenize call. We can discuss this in tomorrow's meeting if I have yet to find a solution.

Fyi @jorpppp.

[rayhuang11]

Things to discuss:

• Bug 1: the local 'option' was including the actual option string (e.g. noci), which was part of the problem. I remedied it by just resetting the local 'option' to "".
• Bug 2: For options that work with the noci(ci noci) syntax, for the single model case, the user must specify the option as xteventplot, noci(noci).
• Implementing options like scatterplotopts

[jorpppp]

Thanks @rayhuang11. Per today's call:

• We'll work on making the ci syntax more intuitive. For one model, the noci option should work. You should not need to write anything else. For many models, the syntax should be ci(#) where # is a list of the models where you want the cis active. So, if you write ci(1 2)then only the confidence intervals for models 1 and 2 out of the plotted models appear. If you write ci() then all the cis disappear. noci should be equivalent to ci(). If you specify ci(1 2 3) but you have only two models, then there should be an error message and an exit. The default should be to show all cis.
• supt should work the same way as ci
• We should generalize ciopts to work as a repeated option that modifies each one of the ci graphs for each one of the models. For example, ciopts(1, color(red)) ciopts(2, color(green). Same with suptopts.
• scatterplotopts should work the same way.

[jorpppp]

There should be some entry about repeated options in the manual section for syntax

[rayhuang11]

Final status update

For the final state of the code, please see the issue7_xteventplot_test.ado file under the issue_7 folder.

What works:

• Currently, xteventplot is limited to plotting 6 models. This can be easily adjusted around line 50.
• Options noci, and nosupt work as repeated options, as in the following syntax: `xteventplot model1 model2, ci(ci noci) supt(supt nosupt)
• Options nozeroline apply to all the plotted models.
• Levels apply to all models.
• P-values, overlays, y, proxy, smpath, override, overidpost, smplotopts, staticovplotopts, trendplotopts, addplotopts, and textboxopts are disabled.
• The offset option works and the default is set to 0.2.
• Appropriate error messages have been implemented for disabled options.

Not implemented:

• Options scatterplotopts, suptplotopts, ciplotopts, and suptreps have not been implemented.
• The ci syntax described in https://github.com/JMSLab/xtevent/issues/7#issuecomment-1237403061 has not been implemented.

Things to note:

• As described in https://github.com/JMSLab/xtevent/issues/7#issuecomment-1237377600
• Some tests have been added to test.do, but more should definitely be added.
• There may be an issue with the nozeroline option.
• Around line 646 I dealt with a graphing bug by manually setting the problematic local to empty.

As of the posting of this comment, I am not planning on doing any more work on this issue. Of course, feel free to reach out to me if anything seems unclear. Thanks!

[jorpppp]

Thank you @rayhuang11 ! There's been a lot of progress in this issue. @Constantino-Carreto-Romero and I can finish this up. We'll let you know if there are any questions.

[jorpppp]

Pending, @jorpppp has to check.

[jorpppp]

Pending, @jorpppp has to check.

[jorpppp]

I managed to do some basic testing of the modified xteventplot and I glanced at the code. There are a few things that we should fix before tackling the options that are listed as not implemented in https://github.com/JMSLab/xtevent/issues/7#issuecomment-1239905659

• [x] The default should be having different colors and markers for the point estimates in the different models
• [x] The normalized coefficient should not be plotted many times, if it coincides across models
• [ ] The default offset (0.2) seems small.
• [ ] We should be able to have different offsets between models when there are multiple models
• [x] The current version for testing in this branch still has the issue from #105, so maybe it's a good idea to bring this branch to date and enter the changes in xteventplot.ado (right now they are on a separate file in the issue folder)
• [ ] The overlay option is not working with a single model now, because the additional graphs are stored as overlays
• [ ] There's no need to tokenize the model names to loop over them

I'll try to work on these a bit, next week we can revisit with @Constantino-Carreto-Romero to pass some of these tasks to him.

[jorpppp]

The branch for this issue was brought to date in https://github.com/JMSLab/xtevent/commit/e326a4b47f164d660dbbbacb7399a64e48997f8e

[jorpppp]

@Constantino-Carreto-Romero will check this after #59 for Monday's meeting.

[Constantino-Carreto-Romero]

Update: I have already started to revise this issue.

[Constantino-Carreto-Romero]

• [ ] The default offset (0.2) seems small.
• [ ] We should be able to have different offsets between models when there are multiple models

Regarding those points: the default offset of 0.2 looks ok with two or three models, but for more models, the graph looks cramped. Maybe we could change the default offset to be more flexible. I think a simple rule could be to set offset equal to 0.n - 0.1, where n is the number of models. I'm going to test that rule and bring some examples.

[Constantino-Carreto-Romero]

in https://github.com/JMSLab/xtevent/commit/79405d6ab316628f06dfdb430c6d756125387e4e I changed the default offset, so now the point estimates for a corresponding x-axis label are placed according to the following:

• the space to place the point estimates is delimited by -0.5 and 0.5 around the x-axis label.
• the point estimates are placed so they are separated evenly and centered around the x-axis label.

This was the previous rule for the offset. I changed it for this rule.

in https://github.com/JMSLab/xtevent/commit/cee873814d6a65c4cfa298fc5f84c7113a7ab049 I create plots with 3 models and with 6 models. These are the plots with the previous default offset: 3 models original, 6 models original. With the new rule for the default offset: 3 models, 6 models. Currently, the offset option works for two models only, and its input defines how much distance the second point estimate will be to the right of the first point estimate. I think we could modify this option as well.

[Constantino-Carreto-Romero]

@jorpppp

• in https://github.com/JMSLab/xtevent/commit/6dee75a8700ae7d033ab6e89affae825358f6eaa, in this section https://github.com/JMSLab/xtevent/commit/6dee75a8700ae7d033ab6e89affae825358f6eaa#r93203053, I disabled the offset for the normalized coefficient. In this section https://github.com/JMSLab/xtevent/commit/6dee75a8700ae7d033ab6e89affae825358f6eaa#r93203187, I added code for the default appearance (color and markers) of the different plotted models.

• in https://github.com/JMSLab/xtevent/commit/8d7b5661020b9bbb0de1c97e61f9e025de7a841d I modified the code for the models' default appearance, so it now adds default color and marker type when not specified, but keeps applying other appearance options specified by the user. These are the plots produced with the example code I had uploaded to compare multiple model plots: 3 models and 6 models

[Constantino-Carreto-Romero]

Per call:

• The point estimates should be delimited by -0.4 and 0.4 around the x-axis label (instead of -0.5 and 0.5) to leave some space between the groups of point estimates.
• Check plot appearance with many models and few (many) periods (e.g., check if point estimates are distinguishable enough or they are too piled up). This will help to decide the number of maximum allowed models.
• The offset option will be depreciated. Instead, we will implement the spacing option. This option must alter the spacing among point estimates (i.e., how grouped they are).
• Implement an option that clusters the point estimates. For instance, in a plot with 4 models, the user might decide to group the point estimates into two clusters: in the first cluster, models 1 and 2, and in the second cluster, models 3 and 4. Therefore, the space between model 1 and 2 is smaller than the space between models 2 and 3.

[jorpppp]

@Constantino-Carreto-Romero Just checking to see if there's any progress on this.