Stock-taking

[jmshapir]

In advance of our next call I'm wondering if we can see a summary of where we stand on this project.

I'm flexible as to format, but one option could be to mark up the design doc and maybe use color coding to indicate what is done (green), what is in progress (yellow), and what is not started (red).

Thanks!

[veli-m-andirin]

Thanks @jmshapir!

@nateschor @rcalvo12 I will prepare and post the document suggested in https://github.com/JMSLab/eventstudyr/issues/8#issue-1224636945 soon, and you can then make suggestions/edits as needed. Of course, let me know if you disagree.

[veli-m-andirin]

@rcalvo12 @nateschor in 85b9f23 I added a colored version of the document. I had a hard time distinguishing LyX defaults for green and yellow, so I adopted the following code:

Green: Done. Orange: In progress. Red: Haven't started.

Please take a look over the document and make any changes as you see fit. I haven't "updated" the design document in the sense that there are functions that we added/modified later. (For instance, instead of "PrepareLeadsLags" we have two separate functions "PrepareLeads" and "PrepareLags", or currently in #4 we are building a helper pre-cleaning function before making the plot. Since this design document is a consensus, I didn't want to start the challenge of building a new "constitution" as the changes we made are minor.)

Please confirm when you are done reviewing/editing!

@jmshapir @santiagohermo fyi.

[jmshapir]

@veli-m-andirin in case useful, I had in mind using Adobe comment tools to color code. But feel free to use whatever works best, thanks!

[nateschor]

Please confirm when you are done reviewing/editing!

Thanks @veli-m-andirin! I have finished editing in 8c8d3ff. For functions that I have been working on that are colored orange, I wanted to add some color (pun not intended) so I also included additional notes regarding:

• what parts of the function are finished and what still needs to be done (I figured there was a decent amount of variation among functions that are all considered in progress)
• whether I should be able to complete the remaining steps for the functions I have been work on on my own, or if some help would be beneficial.

@rcalvo12 @santiagohermo fyi

[rcalvo12]

@veli-m-andirin I added a couple of very brief notes for context on my work-in-progress functions. Both are done (Might make a small edit to TestLinear) and now just need to work on unit testing.

[santiagohermo]

I'll note here that I had a zoom call with @veli-m-andirin where we agreed in the following:

1. I will install the package and familiarize with how the current version works
2. I will go through the paper again, especially Suggestion 6
3. I will start to work towards either implementing Suggestion 6 or the Freyaldenhoven et al (2019) IV estimator

fyi @jmshapir

[veli-m-andirin]

Thanks @nateschor @rcalvo12 @santiagohermo!

@jmshapir, here is the pdf that tracks our progress. @nateschor @rcalvo12 and I added some notes about the ongoing work, some of them were in the source code and I added them to this PDF as Adobe comments.

My prediction is that, after @santiagohermo joined the team, our chances of coming up with a first version that includes more of what we want, such as the smoothest path implementation and FHS, increased. I still wonder if it would be a good idea to set a time frame for a second stock-taking, after which we would go into preparing a first version with what we have at that date.

That being said, I now see my "winter mode" starting in August to be less of a concern since it looks like we will be achieving more than what I initially predicted. If @nateschor @rcalvo12 will continue to have bandwidth to work on this project for a while after August, I am confident that they will successfuly lead us up until CRAN submission as well, so we may not need to hurry too much.

[santiagohermo]

I've been looking at Section 2 of the paper and playing with the master version of the function EventStudy @jmshapir @veli-m-andirin. I have a question and a couple of comments.

First the question. Following https://github.com/JMSLab/eventstudyr/issues/8#issuecomment-1118853484, I'm trying to understand Suggestion 6. Maybe there is some reference that I could look at here (or a claims note). If so, please let me know what it is!

• Does the Wald Region CR(delta) correspond to the hyper-dimensional cube comprised of the cartesian product of the uniform bands of each of the event-study coefficients?
• k is just the event time here, right? Thus, for each event time we have dim(v) terms that must have the same coefficient.
• Is equation (6) saying that we want to minimize the squared sum of the elements of v?

Comments about the EventStudy function

• I think it would be useful to name the elements of the main return list. Maybe the first element could be named coeftable and the second one arguments?
  • I want to be able to search for results by results$coeftable as an alternative to results[[1]], or results$arguments$estimator instead of results[[2]]$estimator.
  • Also, I'm not sure if having nested lists here is necessary. Maybe a single list where the first element is the coeftable would suffice.
• I think it may be useful if the main coefficient table had a rel_time variable that equals event time. It would also be helpful to rename other variables to names that don't use spaces, such as coef, sd_pointwise, sd_uniform.
  • You could also have a rel_time_label here. If rel_time runs from -5 to 5, then this variable would be -5+, -4, ..., 4, 5+
  • One could also include a row with the normalized value that would have all zeros for standard errors and missing for t-stat and p-value. In my experience, this simplifies plotting greatly.
• Following Suggestion 5, should, overidpost and overidpre have default values 1 and pre + post?

[veli-m-andirin]

Thank you @santiagohermo! I think it would be great to tackle these questions in a new issue. I opened issue #9 for that purpose, and will try to answer them there. (I think we completed most of the other functionality in #4, so I think it would be nice to wrap it up without waiting for the smoothest path implementation.)

[jmshapir]

@santiagohermo @veli-m-andirin thanks both!

If questions come up in #9 that you aren't able to resolve between you, of course just let me know.

@veli-m-andirin in terms of stock-taking, how about if we plan to check in again once everything currently in orange is complete?

We can either leave this issue open until then, or close it and reopen when we're ready for another round of stock-taking.

[veli-m-andirin]

Thanks @jmshapir! I agree with checking in again once everything currently in organe is complete. I think we can keep this issue open.

[jmshapir]

@rcalvo12 @nateschor @ew487 this item is on the roadmap:

Write project Wiki to detail our workflow.

Is it accomplished by this readme?

If so I will remove it from the roadmap.

Thanks!

[nateschor]

@rcalvo12 @nateschor @ew487 this item is on the roadmap:

Write project Wiki to detail our workflow.

Is it accomplished by this readme?

If so I will remove it from the roadmap.

Thanks!

@jmshapir yes the readme accomplishes the task, please remove it form the roadmap. Thanks!

[jmshapir]

@nateschor done, thanks!

[jmshapir]

@rcalvo12 @nateschor @ew487 our plan for this issue #8 was to check back in once everything in orange here was done. I think we've reached that point (and also accomplished some of what is in red, too).

I think the main remaining step to fulfill our original ambition for the first public release of the package is #9.

Therefore, I propose to close this issue #8. Let me know if any objections. If I haven't heard any by roughly this time tomorrow I will proceed.

Thanks!

[jmshapir]

Summary: In this issue we took stock of our progress on the package.

Links for reference:

• Issue branch
• Issue subfolder
• Design document with color-coding for progress