Review notes from Jan 17 2020 meeting, under Data-and-code repository Wiki

[vob2]

Please review and add missing parts, if any

[harishk05]

Looks good to me, expect one point. The way I understood it, the government treated all firms the same before (approx) 2010 -- except for a small subset of "disadvantaged" firms. After 2015, the small firms were paid early and the large ones were paid as usual (late). We may need to discuss and clarify.

[vibhuti6]

Thanks, Vlad and Harish. One clarification: the Department of Defense implemented the policy of accelerated payments in a staggered manner. See the timeline below for an illustration:

If we consider only the time period before 2010 and after 2015, we may run into some identification challenges. This is because Aug 1, 2014 onwards, all contractors were receiving accelerated payments. We, therefore, do not have any control group to capture the causal impact on operational performance in this time period.

[vob2]

Yes. The idea is that before 2011 small and large contractors were treated differently. After 2014 they were treated the same. We are using this for identification.

We can think that we are quantifying the effect of expedited payments on large contractors, while using small contractors as the control.

Would this not work?

[vibhuti6]

Got it, thank you for clarifying. So we will be using small disadvantaged businesses as a control group? My understanding is that before 2011, only a subset of small businesses - the disadvantaged ones - were receiving accelerated payments. And that other small businesses and large contractors were treated the same before 2011.

[vob2]

Yeah, we are going to have to make the parallel trends assumption between control and treatment groups. This becomes less and less believable as the two groups become very different.

But that's the best we have at the moment.

Any ideas for alternative specifications?

[JNing0]

Let me share my thoughts/concerns.

For comparing large and small businesses, I agree with Vlad that the parallel trend assumption may be hard to defend. In addition, maybe there is some interaction between accelerating payments to small and large businesses. Roughly, the payment to small and large business comes from the same pool. So the effect of quickpay on small businesses before and after large businesses are added might be different and we need to be careful to interpret the results.

Another concern is my old question about how well quickpay is actually implemented. As we don't have data on that, we are mainly relying on Barrot and Nanda (2019). But that paper verifies that the implementation of quickpay to small firms in 2011. It is silent about the effectiveness of the later inclusion of large businesses and the associated effects of small businesses.

One thought I have to resolve these issues is to use a different control group. Barrot and Nanda (2019) find that the small businesses with fixed price contracts are significantly impacted by the reform in 2011. However, small businesses with cost-plus contracts are not. They are already paid within 15 days before the reform and experience little or no acceleration on average. So shall we use the small business with cost-plus as control and the ones with fixed-price as the treatment group?

Jie Ning //////////////////////// Assistant Professor Department of Operations Weatherhead School of Management Case Western Reserve University Cleveland, OH 44106 e-mail: jie.ning@case.edu tel: 216-368-3841 ////////////////////////

On Tue, Jan 21, 2020 at 10:17 AM vob2 notifications@github.com wrote:

Yeah, we are going to have to make the parallel trends assumption between control and treatment groups. This becomes less and less believable as the two groups become very different.

But that's the best we have at the moment.

Any ideas for alternative specifications?

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/QuickPay-Operational-Performance/Data-and-code/issues/2?email_source=notifications&email_token=AOBDD5TWJA24NKJHTQZ6L2TQ64GZRA5CNFSM4KIWTNH2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJQDAII#issuecomment-576729121, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOBDD5SNLWHVOIUE2NSOODTQ64GZRANCNFSM4KIWTNHQ .

[harishk05]

Thanks for the helpful discussion and clarifications.

[vibhuti6]

Thanks, everyone.

I agree that the parallel trends assumption may be hard to defend if the two groups are very different.

Using cost-plus contracts as a control group may not work very well because DoD uses cost contracts only when fixed-price is not feasible. In other words, a contract under cost-plus pricing may be performing a very different type of work than that under fixed-price.

One good feature of this dataset is that we observe the type of task being performed. For each contract, we have a variable called "product_or_service_code" that describes the task category. There are ~2,000 such codes. We can use this to match the contracts under treatment and control groups.

We could also just focus on small businesses (at least for a preliminary analysis). The payment terms were different for disadvantaged and other small businesses before 2011. And it was the same after April, 2011. So we could compare the performance of contracts given to these different types of small businesses (for the same task) before and after 2011.