Responding to Referees

[derekwu92]

Hey Jonathan- hope your travels to Singapore went well! Super sorry for the delay in replying - I've been caught up with a bunch of other deadlines and wasn't able to carve out time to work on this until this weekend.

In any case, I have a few updates. Thought I would open a Github thread because it's easier to show figures this way in case we want to go back and forth a bit before formalizing responses to referees and edits (also - I don't have edit access in the Google Doc?).

First, the lower-hanging fruit:

Addressing R2's main comment: When we expand our sample to include HHs in the survey for shorter periods of time (meaning we have fewer quarters pre- and post-receipt), the results look essentially the same. Here are the trends in market income for different sample sizes (now made q=-2 the omitted period rather than q=-4 given the shorter pre-periods for the larger samples):

SNAP: Medicaid: UI:

Related to the point above, R2 had some concerns about why the average age of WIC recipients was so high (43.96) and only 47.3% have kids between the ages of 0-4 (perhaps driven by sample choice?). I have a few answers after digging into this. On the age issue, this is the average of the household head (who isn't necessarily the parent receiving WIC; could be a grandparent, for example). When limiting to the receiving parent, the average age falls to I think the mid-30s. The low share of kids 0-4 is also due to the fact that this is measured during the first month of receipt, and WIC recipients can start receiving benefits when they are pregnant. The share increases to about three-quarters if we measure having a kid 0-4 at any point in the 16 months after initial receipt. It is not 100% because kids may live in a different household at the time of interview (low-income families may have more complex lives). Figure we can discuss some of this in a footnote in the main text.

In terms of controlling for the state-level unemployment rate, we were already bringing it in at the year-month level. So there was a typo in the main text when we said that it only varied at the annual level.

R2 also had a comment about potentially adding some age restrictions, as patterns may differ for those above and below 65. What are your thoughts about limiting the sample to include only heads below the age of 65? We could alternatively keep our sample as is and have some results in the appendix breaking down results by below vs above 65. In general, the changes are more muted for elderly households (with having kids being a bit of an outlier, perhaps because the baseline means are so low):

SNAP: Medicaid:

Next, here are some thornier issues:

R1 had a seemingly innocuous suggestion about including month-fixed effects (to account for seasonality of receipt), but the results actually changed quite a bit more than I expected. In particular, the patterns are more muted. Not sure what to make of them (didn't expect there to be so much seasonality in receipt) but the results are below. Note that the rest of the results in this thread are OMITTING month-fixed effects (unless noted otherwise).

Market Income: Post-Transfer Income: Any Employed: Any Disabled: Has Child 0-4: Any Separation:

Related to the above issue, the results for "any employed" actually disappear for the most part (except for UI) when including month-fixed effects. To be fair, having a value of 0 for "any employed" requires no one in the household to be working, which may be more stringent of a criteria that we want. Alternatively, we could define "any not working" (i.e., either unemployed or not in labor force but between the ages of 25-59) and here are these results INCLUSIVE of month-fixed effects:

Finally, in regards to better connecting to the existing literature, my RA put together a longer document summarizing findings in other papers as well as a shorter summary document. They are attached below. A number of papers look at employment and having kids as triggers, but fewer look at disability and separation. It does seem like the literature tends to be a lot older (for example, most papers using the PSID or SIPP look at data from the 80s or 90s; no paper appear to use the 2008 Panel or later) and there also doesn't seem to be any distinction between recession vs non-recession periods. So those are potentially two avenues we could go down. I did some breakdowns first by SIPP panel (1996 Panel vs 2001-04 Panels grouped together vs 2008 Panel vs 2014 Panels). Don't see super compelling differences at first glance but you might notice some patterns I don't:

SNAP: Medicaid: UI:

Alternatively, I divided results into recession vs non-recession. Specifically, I classified an observation as corresponding to "recession" if any date in the pre-receipt period overlapped with 3/01-11/01 or 12/07-6/09 (official dates of 2001 and Great Recessions). Here, there does seem to be some more pronounced differences? (more so for UI and least so for Medicaid)

SNAP: Medicaid: UI: Market income for all programs:

Wu - Program Dynamics Literature (Revised).docx Program Dynamics Literature Summary Write-Up.docx

[jzhang722]

Thanks Derek

• Changing pre/post time, this looks good to me. I think we should just do -2/-4/-6 though to declutter it a bit. It's kind of hard to read with six.
• WIC age: I agree. Probably even just what you wrote for the respones doc would be sufficient too.
• Age restrictions: for programs like SSDI, SSI, it wouldn't make sense to restrict to under 65. But on the other hand it doesn't make too much sense to look at employment and household changes for seniors. What do you think re appendix vs main sample? I don't have strong opinions--probably lean main sample since it's more responsive to R2. The % change having a child being larger for 65+ is counterintuitive to me.

Let me get back to you on the thornier issues, will need to read your comments and your RAs document in depth (at Heathrow airport atm). My initial thought on the recession results: are we sure we want to go down this route and make recession heterogeneity be one of our key contributions? It was merely an off-hand suggestion by R1. As for the results, I'm not sure what to make of them. On one hand, one would expect market incomes to drop more during recessions, but recessions can also induce less severely impacted applicants to apply.. and it's unclear which effect is larger.

[jzhang722]

Do you want to meet on Thursday after our VA DC call or I'm also free all day on Friday.

[derekwu92]

Thanks for these thoughts, Jonathan!

Re age restriction, I could see a case for restricting to under 65 in the main results - especially for our three main programs (SNAP, Medicaid, and UI). And perhaps include elderly results in appendix? SSDI would be natural to actually condition on below 65 (since it turns into old-age insurance when you’re older). I agree that SSI would be a bit more unnatural, so conditioning on being non-elderly would be effectively be focusing on families with disabled children.

Re recession heterogeneity, I’m not necessarily sold either but it sounded like R1 wanted us to have more of a “hook”. I think the natural one is that we have more recent data, but the differences by panel aren’t super pronounced. Another route is recession vs non-recession, but I agree with you that it risks opening up Pandora’s Box b/c things could go in different directions for different reasons. But let’s talk more about this? I could meet at 12 pm on Thursday 3/28 (right after our VA meeting) or at 3 pm that day. Alternatively could do early afternoon (2 pm-ish) on Friday.

On Mar 25, 2024, at 3:49 AM, Jonathan Zhang @.***> wrote:

Do you want to meet on Thursday after our VA DC call or I'm also free all day on Friday.

— Reply to this email directly, view it on GitHub https://github.com/derekwu92/Program-Dynamics-NTJ-Revision-/issues/1#issuecomment-2017404625, or unsubscribe https://github.com/notifications/unsubscribe-auth/A6BN3O6NT6ZVDFVGE252CG3YZ7JKHAVCNFSM6AAAAABFF352CKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJXGQYDINRSGU. You are receiving this because you authored the thread.

[jzhang722]

Let's do 12pm on Thursday 3/28? Thanks!

[derekwu92]

Sounds good! Will send you an invite

On Mar 26, 2024, at 7:58 AM, Jonathan Zhang @.***> wrote:

Let's do 12pm on Thursday 3/28? Thanks!

— Reply to this email directly, view it on GitHub https://github.com/derekwu92/Program-Dynamics-NTJ-Revision-/issues/1#issuecomment-2020233236, or unsubscribe https://github.com/notifications/unsubscribe-auth/A6BN3O7PDAIAV47XWJMJKD3Y2FPHNAVCNFSM6AAAAABFF352CKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRQGIZTGMRTGY. You are receiving this because you authored the thread.

[jzhang722]

Thornier issues:

1. Including month fixed effects

• The point estimates are actually not that different. For market income, SNAP and UI actually seem larger after including the month FE. Medicaid is impacted the most (not surprisingly, I think people enroll in January?) but still not a big difference.
• Take a look at any child for UI and separation for SNAP, the estimates are much larger in the new version.
• The overall patterns (trajectories) are also somewhat different--they look smoother in the new version.
• Obviously confidence intervals are now massive. Is there a way to shrink the standard errors? Increase sample size (non-balanced panel) or cluster at some other level? We could also experiment with quarter of year fixed effects (instead of month)

2. Any not working

• I agree this looks better.

3. Contribution Why don't we try something like this: "Many papers look at employment and having kids; however, few look at other sociodemographic changes such as separations, disability, and health (cite the few that do). Moreover, these papers typically focus on an older period (e.g., Boisjoly et al. 1998 looks at 1960s) and the collection of papers utilize different datasets. Using a unified event study approach approach with a single panel dataset, we study sociodemographic trajectories post-welfare reform."

• So basically the contribution is a) more socioeconomic b) refresh the time period c) unified approach in data and method (event study is also a "refresh").
• My only concern is that maybe it's not aggressive enough and sort of just a rehashing of what we previously had? What do you think?

[derekwu92]

@jzhang722 Thanks for these comments! Yes, you're right that the point estimates don't change that much after including month FEs and if anything are more pronounced in a few cases...it's just the estimates are noisier. I actually tried including quarter-of-year FEs, and it attenuated things even more for SNAP while making things a bit more significant for Medicaid. Removing clustering (i.e., using heteroskedastic-robust errors) has virtually no effect, interestingly. May try seeing what happens when I increase the sample size with an unbalanced panel.

And re contribution, I think the pitch you propose is good but maybe we can go further. I think the referees would think about (b) refreshing the time period as the biggest contribution - mainly including the patterns after GR, which is a recent time period I don't think any paper has looked at. But then we also don't want to get sucked into trying to explain differences in estimates over time, but maybe we can get away by saying that these patterns have continued to persist even in recent years? Maybe we can brainstorm more about this tomorrow...

[jzhang722]

Hi @derekwu92, I was tasked with comparing and benchmarking our estimates with the literature. Based on my reading from what your RA put together (which was very helpful), our estimates aren't easily comparable to the literature.

1. Most of the literature in the 80s and 90s were concerned with transitions off welfare and sometimes back onto welfare. These include Blank and Ruggles 1996, Stevens 1999, Hoynes 2000. Bane and Ellwood 1986 also look at transitions out of poverty spells (and onto). This literature is just more concerned with how to get people off welfare/poverty.
2. Some other papers look at P(welfare | characteristic) which is flipped from what we study P(characteristic | welfare). For example Grogger 2004 looks at P(welfare | child), but we are studying P(child | welfare), so without knowing P(child) and P(welfare), cannot compare.
3. A bunch of other papers also look at the impact of some shock on outcomes, again flipped from what we are studying. These shocks include job displacement, evictions, hospitalizations, etc.
4. A larger literature study the impact of programs on outcomes (Medicaid Dague et al. 2017, Buchmueller et al. 2021, SSDI Deshpande et al. 2021, SNAP Hastings & Shapiro 2018). I know many referees asked us to cite these papers and I can see how the panel aspect is related to ours, but really they're a lot further from our paper than the older literature.
5. The two most closely related papers to ours are Myers and Mok 2019 and Deshpande et al. 2021. Myers and Mok 2019 finds drops in earnings, pre-transfer, post-transfer incomes of 12.7%, 7.1%, and 5.2% in the year prior to (self-reported) disability onset (all disabled). Our estimates for DI are 12.5% for market income, +7.0% for post-transfer (opposite signed to Myers and Mok). Deshpande et al. 2021 look at rates of bankruptcies, foreclosures, and evictions before SSDI application (outcomes we don't observe)

Takeaway: We should definitely compare our estimates to Myers and Mok 2019 in the paper. Do you think I've missed any papers that look at P(characteristic|welfare)? What do you think about making part of our contribution that we're flipping around what the literature has typically looked at?

[derekwu92]

Hey @jzhang722 - thanks so much for this, and sorry for the slow reply. I finally had a chance to also go through the literature and think a lot of your thoughts make sense, though I may have a slightly different perspective on a few things.

My basic take on the literature (which can perhaps be fleshed out more in a new "Section 2" of the paper):

• An old literature on training programs finds a dip in participant earnings in the lead up to participation (Ashenfelter 1978, Ashenfelter and Card 1985, Card and Sullivan 1988, Heckman and Smith 1999). Suggests that changes in life circumstances may precede enrollment in other programs as well.
• A slightly less old literature focuses on the determinants of the start and end of poverty spells (e.g., Bane and Ellwood 1986, Blank and Ruggles 1996, Stevens 1999, McKernan and Ratcliffe 2005). A related set of papers looks specifically at the individual-level determinants of program participation spells (Blank and Ruggles 1996, Boisjoly et al. 1998, Cancian et al. 2002, Grogger 2004), which is closer in spirit to our paper - it is worth drawing a distinction between the events precipitating poverty and the events precipitating program participation. There are several things worth noting about this literature. First, it tends to use very old data - often from the 80s and 90s (pre-welfare reform) - when the safety net looked very different. Only one (McKernan and Ratcliffe 2005) uses several years of post-welfare reform data and even then they can only look through the 1996 SIPP Panel (which goes until 1999?). Second, the studies that examine program participation focus almost exclusively on one program: cash welfare (formerly AFDC, now TANF). Third, the determinants of poverty or welfare spells are sometimes measured from a static perspective - but even when these predictors (number of children, marital status, job loss, etc.) are measured dynamically, it is usually in the form of a summary measure (e.g., any change over a pre-period of 1 or 2 years) that masks more granular patterns in the pace/nature of the change. Bane and Ellwood (1986) and Boisjoly et al. (1998) fall into this category. On the other hand, McKernan and Ratcliffe (2005) analyze the dynamic determinants of poverty entry (e.g., child under 6, loss of employment, separation, disability, etc.) over a very short window - only 1-2 months pre-poverty.
• To respond to your comment above: it does seem like a lot of prior studies look at things from a perspective of Pr(welfare | characteristic), which differs from Pr(characteristic | welfare) - although it should be noted that in the latter specification we'd be focused on pre-enrollment characteristics rather than post-enrollment characteristics. I don't know if focusing on the latter is a huge contribution, but I do think it is a cleaner/more straightforward way of investigating the changes in circumstances that lead to program enrollment. This is because looking at Pr(welfare | characteristic) is conditional by construction on a broad group of individuals (e.g., those who are eligible or even broader) and so the changes in life circumstances may have less predictive value given that they have to also compete with other determinants of participation like administrative burdens. Looking at Pr(characteristic | welfare) focuses on those already receiving a program, for whom we may see sharper changes in life circumstances - and ultimately we do want to select on program recipients as our sample given that we're focusing on the reasons leading up to program participation.
• A number of studies look at more macro or policy determinants of program participation (Hoynes 2000, Gittleman 2001, Ribar et al. 2008, Ganong and Liebman 2018). This is of course related to our paper, but it is also fundamentally distinct from what we are looking at because we are focused on individual- and household-level changes.
• There are some more recent studies that do look at events leading up to bad events or program participation using a framework that is similar to ours (i.e., dynamic patterns in characteristics around program application/enrollment). However, a lot of these results are buried as minor results within broader papers. Meyer and Mok (2019) and Deshpande et al. (2021) are two closely related papers as you note - although the former looks at events leading up to disability (rather than take-up of disability programs per se) and the latter looks at financial and housing outcomes that we do not analyze. Collinson et al. (2023) is another paper showing that earnings and employment dip sharply in the months leading up to eviction (Figure 2). There are some other papers focusing on program enrollment that we may want to cite and benchmark results against: Dague et al. (2017) show reductions in earnings prior to applying for public health insurance (Figure 1), Hastings and Shapiro (2018) show reductions in earnings and increases in having child under 5 prior to SNAP adoption (Figure 1), and Cook and East (2024) show reductions in earnings (driven by full-time employment) prior to SNAP application (Figure 1). That being said, these three papers all look at results within a single state (Wisconsin for Dague et al., Rhode Island for Shapiro et al., and an unnamed mountains plains state for Cook and East). In addition, the focus on different outcomes, programs, locations, and time periods across these papers makes it hard to do apples-to-apples comparisons.

So given this background, I think there are several contributions of our paper that are worth highlighting:

• We look at the safety net in the modern era, which is much more heterogeneous and broadly accessible than what existed pre-welfare reform. Much of what we know about trigger events is in the context of AFDC, which targets only low-income families with children and is very different from the diverse array of programs we have today.
• Within this contemporary context, we use a unified framework to compare and contrast patterns around multiple programs for the entire U.S. This provides a more thorough look than the recent studies that do some of these analyses in the modern era, as they tend to focus on a single program (e.g., SNAP), a single state, and earnings/employment outcomes. We are able to go beyond just looking at labor supply and can assess outcomes such as marital separation, which studies - after those from the 1990s/early 2000s on AFDC - have largely ignored.
• We also look at granular changes in outcomes (at the quarterly level) that paint a richer picture of the pace and nature of events precipitating program enrollment, compared to what you might learn analyzing "any event" during an X year period prior to participation. This is useful e.g. for assessing whether program participation follows a sudden or protracted decline in some characteristic, and whether these "dynamic" patterns differ across programs. Thoughts on this?

Finally a quick update on the other results...sorry for the delay in getting back to you on them, but I hope to have revised results outputted in the next day or two. The key update is that when we use an unbalanced panel, which increases our sample size by a lot, we are sufficiently powered for our regression estimates even after including calendar month FEs!

[jzhang722]

An old literature on training programs finds a dip in participant earnings in the lead up to participation (Ashenfelter 1978, Ashenfelter and Card 1985, Card and Sullivan 1988, Heckman and Smith 1999). Suggests that changes in life circumstances may precede enrollment in other programs as well.

You are right. To the extent that training program can be compared to welfare programs, I totally agree. I think they are comparable, but not perfectly. These training programs are about improving human capital and future gains, whereas welfare programs are more about improving levels.

My comment about P(characteristic|welfare) is trying to highlight 2 things. First, it means we can't compare our estimates (which was at least my objective) and second, is what you said, we are focusing on those participating in programs. Whereas these other papers are interested in getting on/off programs.

A number of studies look at more macro or policy determinants of program participation (Hoynes 2000, Gittleman 2001, Ribar et al. 2008, Ganong and Liebman 2018). This is of course related to our paper, but it is also fundamentally distinct from what we are looking at because we are focused on individual- and household-level changes.

I don't see these as related enough to mention in detail for a literature review. To me the Collinson, Dague, Hastings Shapiro, etc. papers are even less relevant. Collinson is looking at evictions, Hastings and Shapiro + other papers are looking at impact of program on outcomes, and the Dague et al. is the most related since they show that there is reduction in earnings prior to going in health insurance, but that is not the main point of the paper.

I really like how you summarized the 3 bullets for our contribution. I'd like to add to the first bullet that these older literature typically focus on static characteristics and many are concerned about getting off welfare or out of poverty. These older literature also compare characteristics across participant vs non-participant, whereas we are making comparisons within a participant (technically cohort..) across time. These are points related to your third point, but I think what we are doing is fundamentally different from (most of) the older literature, even if they "feel" the same.

[derekwu92]

@jzhang722 Thanks - this all makes sense. Your addition to the first bullet sounds great as well - sorry I had meant to add something about the fact that the older papers are often concerned about poverty/welfare exits (since you had identified that) but it slipped my mind. Re the more recent papers, I agree that their main focus is far afield from what we are looking at, but I wonder if it is worth still briefly discussing at least Hastings and Shapiro, Dague et al., and Cook and East. In some ways, their results on changes before program receipt are most comparable to ours in terms of benchmarking estimates (especially Hastings and Shapiro and Cook and East on SNAP). But more importantly, the fact that they are relegated to a minor subsection of a given paper points to the paucity of relevant estimates in the modern era - further emphasizing the need for a paper like ours to fill this gap.

In other news, I have updated nearly all figures in our paper - see here (also on Overleaf): Program_Dynamics_in_SIPP_and_VA-7.pdf

There are several key methodological changes throughout:

• Control for calendar month FEs in addition to year FEs (additively)
• Use unbalanced panel, which ends up increasing the sample size a fair amount and gives us enough power even after including month FEs
• use household rather than cohort FEs; this is a new change - previously we've been using cohort FEs and the results are generally similar between the two (point estimates are maybe slightly smaller using household FEs). However, the results on disability and separation are for some reason much more powered using household FEs, especially after including calendar month FEs.
• changed the employment graphs to "not working" throughout (where not working is defined as at least one member either unemployed or not in the labor force but aged 25-59)

One thing to note is that after bringing in these changes, there is a bit more divergence in the patterns depending on the choice of sample (see Figures C4-C6; more pronounced for SNAP).

Finally, along the lines of contributions to literature, I wonder if we may want to include two additional figures to the paper and perhaps discuss them at the end of the results section. The first would show changes in market income by family type (elderly vs non-elderly with children vs non-elderly without children); given that the prior literature has focused mostly on families with children, it would be useful to see if similar or different patterns materialize for other groups. The second would show changes by pre- vs post-08. We probably wouldn't want to go down too much of a rabbit hole in terms of making sense of these results, but we could speculate briefly on them and that would at least be somewhat responsive to R1's comments. In any case, here are these two figures:

[jzhang722]

Thanks @derekwu92 I think we're almost there!

I like the new figures. Many of them look much better (for example TANF is significant now). A few comments on this:

• R2's Comment 1:

  1. The paper should include more discussion of how sample selection might be influencing the patterns documented. To make it into the sample, a household must be continuously present in the SIPP for at least 36 months, which corresponds to 9 waves of the SIPP. I worry that this creates an analysis sample that is more geographically stable, with higher and less volatile incomes, and with fewer changes in household composition, relative to the full population of recipients of safety net programs. I suggest that the authors try a version of the analysis that requires fewer observations per included household (maybe just 9 months before and 9 months after program receipt), and that they add discussion about how this affects the results.

Our sample is now completely unbalanced right? Is there anything like X months before and X months after? We're sort of in the spirit of addressing this comment.

• Aesthetically, would it be a lot of work to extend the x axis out in Figure 2a and Figure 4a-c to have a bit more white space beyond x=0? The x axis limit is too close to zero that the vertical line is hard to see.
• Robustness check C4 for SNAP.. I don't know what to do. I'm sort of confused why the balanced vs unbalanced makes such a big difference. Let's say we leave it. Referees almost certainly will not check these.

I think the main things to do now are to write up the contribution for introduction. Can you do this? I feel you have a better grasp than me at this point. I think it makes sense to have it be a little showy and on the more detailed side just to show that we've taken it seriously. They might come back asking us to trim it down and that's fine.

Could you take a look at the VA part that I did a month back (or maybe you already did and I missed it). I also left a couple comments.

Re: final 2 figures you produced. I'm not sure. One option (and this is how I'm leaning) is to write up the contribution as you suggested in https://github.com/derekwu92/Program-Dynamics-NTJ-Revision-/issues/1#issuecomment-2053731992, and then in the response document under the part where the response to editor and R1's " another option would be to examine whether and how these associations changed after the Great Recession." We can say we looked at pre/post GR and also by household composition (include the figures in response doc). We do think this is a new contribution, but not super informative. We're happy to include it in the paper if you'd like us to.

Shall we schedule a call for next week (after we iterate once on the intro) to go through the response doc together?

[derekwu92]

@jzhang722 Thanks for these thoughts! Some quick responses:

• Yes our sample is now completely unbalanced. The only restriction is that we must observe at least one month prior to initial participation. Could think perhaps varying this restriction a bit to observe e.g. at least 2 quarters, 4 quarters prior to enrollment while keeping sample unbalanced (as a robustness check). I'm also not sure about why the balanced vs unbalanced trends are so different for SNAP in Figure C4, but it does at least seem like the differences are most pronounced in the post-period (which we are less focused on). The pre-period drop is still noticeable even though attenuated.
• Re extending x-axis out in the coefficient plots: yes this should be easy to do this.
• Re writing up contribution: yes happy to do this. I was actually thinking about including a new section after the introduction purely dedicated to a literature review and ending with our contribution, just so we err on the side of being overly responsive. Better to have them say we overdid it than say we didn't do enough.
• Re the VA section: yes I can look over this. Sorry I haven't had a chance to go through it yet.
• Re the final 2 figures on heterogeneity: sounds good re including them in the response doc!

Good idea re scheduling a call for next week. I'm hoping to try and carve out some time later this week and/or this weekend to work on this - how about sometime next Tues 4/23 (free all day) or Th 4/25 (free before 3:30 pm)?

[jzhang722]

Yes our sample is now completely unbalanced. The only restriction is that we must observe at least one month prior to initial participation. Could think perhaps varying this restriction a bit to observe e.g. at least 2 quarters, 4 quarters prior to enrollment while keeping sample unbalanced (as a robustness check). I'm also not sure about why the balanced vs unbalanced trends are so different for SNAP in Figure C4, but it does at least seem like the differences are most pronounced in the post-period (which we are less focused on). The pre-period drop is still noticeable even though attenuated.

I don't feel strongly. But do you think we should make the main specification balanced +/- 9 months before to be entirely responsive to R2? We're being somewhat responsive already by trying an unbalanced panel.

Re writing up contribution: yes happy to do this. I was actually thinking about including a new section after the introduction purely dedicated to a literature review and ending with our contribution, just so we err on the side of being overly responsive. Better to have them say we overdid it than say we didn't do enough.

I'm good with this!

How about Thursday 4/25 at 2pm?

[derekwu92]

@jzhang722 Thanks! I modified appendix figures C4-C6 somewhat so that the sample restrictions now apply to the "unbalanced" panel...specifically in one we restrict the sample to include only households for whom we observe 2 quarters (8 months) of data before participation and in another we restrict the sample to include only households for whom we observe 4 quarters (16 months) of data before participation. Everything else is unbalanced though. Estimates look better:

SNAP (market income):

Medicaid (market income):

UI (market income):

SNAP (post-transfer income):

Medicaid (post-transfer income):

UI (post-transfer income):

We could also do a version where we condition on having some data post-enrollment too (as R2 suggests), but in a way we really just care about the pre-enrollment dip so perhaps it makes sense to condition the sample on having pre-enrollment data. Thoughts?

And Th 4/25 @ 2 pm sounds good! Will send you a calendar invite.

[jzhang722]

Thanks @derekwu92!

It sounds like you prefer to keep the fully unbalanced panel for the main sample and then show these new robustness checks based on restrictions on pre-enrollment quarters. Am I right?

I'm fine with that. I think it's still pretty responsible to R2's comment.

[derekwu92]

@jzhang722 Sorry realized that I never responded to this!

Figures C4-C6 in the latest version of the paper (https://www.overleaf.com/project/60c8d288f5c1979f26e60460) show the robustness checks, and the robustness checks I ended up including are balancing on 2 quarters pre-enrollment, 4 quarters pre-enrollment, and 2 quarters pre- and post-enrollment (latter is fully responsive to R2 - figure might as well just to be safe). But our overall estimates are fully unbalanced (we only restrict on observing one month prior to enrollment).

Also, I've been working on revising the paper and have rewritten the introduction and inserted a literature review section if you want to take a look before our Thursday meeting. Haven't yet gotten to Section 3 and later but hope to do some more work on it before we meet.

[jzhang722]

Thanks! I'll take a look at intro and lit review before our meeting on Thursday.

[jzhang722]

Hi @derekwu92

I've trimmed the literature paragraph in the intro by a little bit. I found it a little long and I'm still not really sure it's needed in the introduction. Most papers with a separate literature review section don't have literature paragraphs in the intro. I also put the citations around adverse non-program events (e.g., Collinson, etc.) in a footnote since I don't think they're all that related.

Section 2 (literature) is also 3.5 pages. I think that's at least 1 page too long so I made some changes with track changes on. I made some minor changes cutting some sentences down. The last three paragraphs need to be more concise. We can discuss this on the call.

[jzhang722]

Hey @derekwu92

I am done with Sections 1 and 2 entirely. Everything is in track changes so please go through them once you get a chance.

I've also filled out more of the response document. Places where I need your attention or have to still do are highlighted.

I think on my end, I need to work on the VA appendix once you go through Section 3. Is there anything else I need to do?

[derekwu92]

@jzhang722 Thanks very much! I'm going to prioritize this paper over the weekend and will look over your edits/incorporate more changes in later sections/edit response doc. Hope to pass something back to you by early next week. If you want to go ahead and try and make the VA appendix self-contained (with a discussion of the program details, etc.), that would be great. I will remove all references to it from the main text (except when we refer to the appendix)

[derekwu92]

@jzhang722 Just an FYI that I've now gone through most of the draft...was hoping to finish it today but I had to make a last-minute trip to the Fed Board RDC yesterday and today (just came back) and ran out of steam. Just have a couple more things to do (finish going through Sec 2, discuss trends in total benefit dollars in Sec 3, and then the VA appendix section) but I'll finish that tomorrow and will also work on the response doc then. In any case, feel free to take a look through what I have so far if you get a chance. Thanks!

[jzhang722]

Thanks @derekwu92 All I need are the VA DC SIPP results and for you to look through the response doc. I'm done with the rest.

[derekwu92]

@jzhang722 Great - thanks. Just finished up full draft on my end and also went through the response doc: https://docs.google.com/document/d/1meN8m0p1gSTrzeqc7xFwpIAK32IJxk9OASq4eYlf_QY/edit.

Look forward to chatting later this afternoon - I think we're close!

[derekwu92]

@jzhang722 Wrapped up my edits to paper and Google Doc so ready to pass it to you! Will be traveling rest of today but happy to get back to it tomorrow (Sun) or early next week.