Things Before Next Meeting

[Page007]

Following things to do before next meeting with Scott:

1. Trimming based on event time: restrict treatment effects to 3-leads/lags while keeping everything else the same.
2. Take the 8 counties that are treated just once with both civil and small claims and check for (a) MH (b) sued.
3. Drop CA from the estimation sample to check if the results are purely compositional.

4. $L(x, y; \theta) = f(y|x)\cdot f(x)$ where $x$ and $y$ are payment amount or recovery rate. Fix a parameter vector $\Gamma$ for the distribution of $f(x)$ and find a closed form for $f(y|x)$. Maximize the likelihood and find parameters of $f(y|x)$. Search over the grid for different parameter values in $\Gamma$ and converge to a $\Gamma$ and parameter vector for $f(y|x)$. But need to figure out a nice closed form for $f(y|x)$.

5. Read more about Tobit -- we observe the payment amount for those who're sued. Using Tobit, we can identify the payment amount for people who're not sued. Then find out the conditional distribution of recovery rate on the payment amounts.

[Page007]

In light of our recent discussion about compositional effects, I decided to briefly go back to our treatment effect estimation. Construct a monthly individual level panel where individuals belong to the risk set (>3 colls and coffs) from a random 5% draw from the raw TU data. Drop all individuals who ever reside in CA (there can be cross-state movers). This is my estimation panel.

In the following graphs, the regressor is small claims fees. It is a bit wild why standard errors are so high in levels specification.

[Page007]

To find the effect of filing fee variation on suit rates, we took the estimation sample to be a risky-individual level monthly panel between 2009-17 that contains a dummy for being sued this month. A borrower is Risky if he has >3 charge-offs and collections in at least 1 month between 2009-17. Then we ran SA on this panel and the results are here. Do note that regressor is Civil Filing Fee in figure notes means that the treatment is the civil filing fee. In what follows, I assume that the reader is familiar with everything written here.

From issue 47, CA contributes 50% counties to our county-level raw filing fee data. When the treatment is civil filing fee variation, all these counties are treated in the 2010 cohort, while if the treatment is small claims filing fee variation, they are in the control group. In the light of these facts, I worry that the elasticity of suit rate to filing fee that we estimated before may have been driven by treatment effects in CA. I elaborate below.

When the treatment is civil filing fee, consider the 2010 cohort. The number of control counties is 8 while the number of treated counties is 25 (only CA). None of the control counties is from CA, but from: GA (De Kalb), IL(Du Page), MO(3 counties covering St. Louis), Hamilton OH, Knox TN and Lubbock TX (doesn't contain any of the major cities in TX). Thus, suit rates in the control group might be trending very differently in pre-periods than in CA. To make matters worse, we observe only 1 pre-period in CA, so it is difficult to observe parallel trends. CA also has post-periods from 2010-17, which means that all post-period coefficients have treatment effects from CA in them. If parallel trends indeed doesn't hold, then all post treatment period coefficients (or treatment effects) we see might be biased. To assuage concerns about parallel trends, I tried dropping CA completely from the estimation sample and re-ran the levels and logs specification. It was purely an ad-hoc analysis exercise, so I picked up codes from different files I've written over the months-- that's the reason for non-standardized graphs.

For brevity, the levels spec. looks like: $sued{it} = \alpha{g(i)} + \alphat + \Sigma{l = -6}^7 \mu_l \cdot \mathbf{1}(t - Ei = l) + \varepsilon{it}$ where $E_i$ is the year of treatment of individual $i$'s county. The ES plots are made by plotting the $\mu_l$'s. Because the treatment varies over years, $\mu_l$'s are estimated at an yearly level and therefore, the X axis in our project is always in years. For example, lead 2 is 2 years before the treatment in an event study plot.

When the treatment is Civil Filing Fee variation, the treatment effect on suit probability levels in an estimation panel that doesn't contain CA counties looks like:

When the treatment is Civil Filing Fee variation, the treatment effect on suit probability levels in an estimation panel that contains all counties from the raw filing data looks like:

The treatment effects seem to change a little bit in the post-periods, SEs change ever-so-slightly but pre-trends remain the same. They should remain the same given that CA counties do not contribute to any leads >1.

When the treatment is small claims fees, CA is always in the control group -- its counties form more than 50% of all control counties. Therefore, if the outcome (suit probability) is trending differently in CA as compared to other control counties, we might have an issue. This is why I dropped CA from the estimation sample and re-ran the analysis -- the event study plot follows.

Happy to get into why the standard errors here are so high compared to the SEs in the event study plots in which the treatment is civil filing fees. But because this is not the point of this post, I refrain from this.

When the treatment is small claims Filing Fee variation, the treatment effect on suit probability levels in an estimation panel that contains all counties from the raw filing data looks like:

[Page007]

This post is re: MH regressions

Hi! I am just listing down a list of to-dos after our meeting on 31st May'24.

• [ ] Run MH regressions on trimmed sample (a-la #50 this particular issue)

• [ ] Run MH regressions on sample excluding counties with both SC and CC filing fee variation once between 2009-17. For example, Cook IL has $39 increase in SC fees and CC fees in 2010: analyze how dropping counties like Cook IL from estimation sample change MH ES Plots.

• [ ] Re: composition effects driving black-encircled patterns in the ES Plots for MH regressions, i.e., estimation of the elasticity of delinquency to filing fee variation (on a panel that contains all counties that are in the raw data):

  

  The black-encircled effect comes only from counties treated in 2010, 2011 and 2012 cohort. As a side-project, I should try running the same MH regressions on: (a) an estimation panel that drops all counties treated in the 2010, 11 and 12 cohorts (b) an estimation panel that only includes counties treated only in 2010, 11 and 12 cohort.

• [ ] We still do not know if the civil filing fee/small claims fee variation is the "treatment of interest" or the "correct fee the plaintiff pays" -- one idea as discussed in the meeting is to find the elasticity of the suit rates/delinquencies/payment amount due on filing fees on a monthly panel of borrowers only from counties that receive: (a) only civil filing fee variation (b) only small claims fee variation. We might get motivating evidence for treating either civil or small claims fee variation to be the "treatment of interest".

[Page007]

This is re: the MH regressions where the target is to find the elasticity of various measures of delinquency (30/60+ or 90+ or some combination of those) to filing fee variation on a monthly individual level panel of borrowers pulled randomly from raw BTU data between 2009-17. In the last meeting, we decided to "trim" to check if patterns like the black-encircled one below are purely compositional:

Before we agreed to trim, you had asked if each of the black-encircled coefficients are treatment effects on different populations and the answer to that is yes.

However, from remark 2 of Schmeidheiny et al., if we trim the estimation sample at finite leads and lags, we need to make an assumption that the treatment effects outside these leads and lags are not trending -- for example, dynamic treatment effects over-time should have stabilized to a level after the last lag and before the first lead. But this assumption clearly doesn't hold in our estimation sample; for example, if we trim at lag_4 in the figure above, the treatment effects are clearly trending and not constant.

However, I think that Schmidheiny and Siegloch give 2 contradictory/incomplete approaches of binning/trimming in paragraph 1 of section 2.2 of their paper. Initially, I believed their first approach here, implemented it and got bizarre results. (happy to elaborate more on what I did and what I found). In this Git post, I said that we should trim our sample because it is supported by first approach in paragraph 1 of section 2.2 of paper. I am sorry I believed them blindly without asking for proof.

So I turned to approach 2 and found that we cannot trim unless we make an assumption of constant effects outside our lags and leads as explained in the paragraph 3 of this post.

Alternative Approach

Observe that the black-encircled coefficients in the figure above are lags 5, 6 and 7. Because time period in our sample ranges from 2009-17 and the first treated cohort is 2010, 5th, 6th and 7th lags are only computed for 2010, 2011 and 2012 cohorts. (7th lag is computed for only 2010 cohort, 6th lag for both 2010 and 2011 cohort and 5th lag for 2010, 2011 and 2012 cohorts). I propose that we drop counties treated in these cohorts from our estimation panel and find the elasticity of various measures of delinquency to filing fees.

Happy to explain more of my reasoning for the alternative approach, but I worry that this post is already very long and might spur some questions, especially in the first 3 paragraphs. It would be great to hear your thoughts on this.

[Page007]

Thanks! b1 is the set of dynamic treatment effects -- for example, if there are 7 leads and lags in the untrimmed sample, we can find unbiased estimators for $[\beta{-7}, \dots \beta{7}]$ using the full (untrimmed sample) sample. But if we trim this sample at [-4, 4], i.e., exclude observations for all treated units outside t = -4 and 4, then $[\hat{\beta_{-4}}, \dots \hat{\beta4}]$ estimated using SA are not unbiased estimators for $[\beta{-4}, \dots \beta_4]$. This is because of OVB, something which I explain towards the end of this post.

So to summarize in your example, b1 is the set of all dynamic treatment effects on the untrimmed sample, b2 is not the set of all dynamic treatment effects on the trimmed sample.

P.S.: if the sample is trimmed, the bias comes in because of OVB. The treatment effects in "trimmed" time periods are correlated with the treatment effects in "untrimmed" time periods. So, if the true model of the world is:

$$y{it} = \alpha{g(i)} + \alphat + \Sigma{l = -7}^{-5} \mu_l \mathbf{1}(t - Ei = 1) + \Sigma{l = -4}^{4} \mu_l \mathbf{1}(t - Ei = 1) + \Sigma{l = 5}^{7} \mu_l \mathbf{1}(t - Ei = 1) + \varepsilon{it}$$

but we trim the sample, we essentially run:

$$y{it} = \alpha{g(i)} + \alphat + \Sigma{l = -4}^{4} \mu_l \mathbf{1}(t - Ei = 1)+ \varepsilon{it}$$ and thus we have OVB if $\mu_l$'s over time are correlated with each other.

[Page007]

You're right, but the population parameters $\mu_l$'s we estimate are for a population in which the treatment effects are zero outside of the untrimmed time periods.