Results for congestion theory

[vibhuti6]

Hi everyone, I looked at the data to test the congestion theory (code here). Please review below and comment, thank you!

Number of small projects do not increase after QuickPay. There's not a noticeable change in trend visually, and the regression indicates the number of small projects have decreased after the reform.

[vob2]

What about the number of projects per firm?

The idea for the congestion is that a firm takes up more projects. But the number of firms can be changing, So we may have congestion, but if the number of firms decreased, the total number of projects decreases.

Also the prediction is that the number of projects per firm will increase, not only small but all. Of course, the firm must have some small projects that are subject to QuickPay Law to have the incentive to take up more projects after the law is implemented.

[vibhuti6]

Thank you, Vlad! I looked at the number of projects per firm.

Here's the plot for full sample:

And here's the plot when we restrict the sample to contractors that were active before QuickPay:

These plots indicate: (i) more projects taken on by existing contractors after the reform, and (ii) entry of new contractors. The first effect seems more pronounced based on the plots.

I ran the regression for the impact of number of projects on delays, the effect is significant in three out of five specifications.

The effect is positive and significant in discrete version of regressions (with all controls & fixed effects).

[vob2]

Great! Thank you!

Certainly looks like there are more projects per contractor. Does it make sense to ask if the increase is the same for treatment and control groups? Because we are doing analysis at the project level, probably not. But maybe you or someone will have an idea.

I guess we could keep this theory. Mentally, I always ignore the first specification. The second is OK, but the real models begin with the third. Too bad, the effect is not there. But that's life.

[vibhuti6]

Thank you, Vlad -- I plotted the number of small & large projects per contractor in the graph below. The increase seems to be uniform across the two groups.

[JNing0]

Thank you, Vibhuti. Am I right that Tables 5 and 6 establish that the delaying effect of QP is stronger for firms with more projects? The link between these tables and the congestion theory is confusing to me. My understanding is that there are two steps to the theory:

1. QP causes firms to take on more projects
2. More projects leads to delays

Note that the QP effect comes in only in part 1. The effect of project number on delay has nothing to do with QP; it is a general result from the queueing theory. (If anything, QP should alleviate the congestion because contractors are paid faster. This is why the result in Tables 5 and 6 is confusing to me.)

Let's run a DiD on the avg. number of projects per contractor and then a simple regression on the number of projects on delays. From the figures you've plotted, part 1 seems true. But we still need a regression. Could you run these regressions on parts 1 & 2?

[JNing0]

Vibhuti, in addition to the analyses we discussed yesterday, i.e., project num/value DiD, delay DiD in general version and stratified version (e.g., quintiles), could you also do an event analysis on the delay wrt project num/value? It would be interesting to see how things change over time. Thanks!

[vibhuti6]

Thanks, Jie -- I will do that.

[vibhuti6]

Hi everyone, I ran the DID for number of projects per contractor, and there does not seem to be any significant change after QuickPay. I ran two versions: (1) restrict sample to contractors holding only small or only large projects, and (2) full sample where contractors holding at least one small project at any point are "treated".

Also, one (possibly relevant) observation: our sample constitutes few large contractors but they each have a bigger portfolio of projects on average (and the opposite holds true for small contractors).

Here are the results:

Case (1)

Case (2)

[vibhuti6]

There does not seem to be an increase in the overall budget or number of tasks either. Please see below.

Budget

Tasks

[JNing0]

Thank you, Vibhuti. I guess that's it for the congestion theory.

But we can still use tables 5 and 6 for the portfolio theory? Here is the idea. The more projects a contractor has, the more likely it will exploit the faster payment under QP and expedite other projects. So delay of treated project worsens as the portfolio grows.

We can still run the stratified analysis (e.g., in quintiles) to check if the portfolio effect depends on the portfolio size. (I'd expect the delay to exacerbate more as portfolio grows.) We can also do the event study to see how the portfolio effect changes over time.

[vob2]

Jie+1. Congestion theory cannot be applied because there is no greater congestion.

There is another way we can take advantage of the analysis Vibhuti did. Considering we have a pretty comprehensive set of figures all pointing out that congestion did not increase, we can still include congestion theory and use these figures to rule out congestion explanation empirically. Although this is a null result, it can be valuable because a priori we did not know whether congestion forces apply.

[vibhuti6]

Thank you both!

One question: Do we have a theoretical explanation for why a bigger portfolio would lead to treated project being delayed? If I understand the existing theory correctly, we argue that greater number of large projects could cause more delays on small ones because of sequencing. Tables 5 and 6 in contrast look at the "total" number of projects.

As such, I agree there is value in keeping the congestion theory and showing the null effect. The results indicate that small contractors are not utilizing the increase in resources/capacity to acquire more projects. Hence, congestion cannot be an explanation for greater delays after QuickPay. This seems interesting and somewhat counterintuitive.

[JNing0]

The way I see it is that there is no fundamental difference between small and large projects. A contractor can prioritize any project as it sees fit, when exploiting faster payments. The more projects a contractor has, the more candidates for prioritizing. It does not need to be large projects only, I think.

[JNing0]

Another direction is explore is to use "number of large projects per contractor" in the regression. This is more in line with the current theory.

Bottom line: let's include all positive results in the paper, as long as we can modify or create theories for them, which I don't think is a challenge as we have Vlad and Harish :)

[vob2]

That is true. We can make up any theory to fit the data :)

[vob2]

Here is one explanation, for example.

The current portfolio theory is that if a contractor has one small and one large project, then when small project's payments is expedited, the contractor sequences the large project first so that the waiting time for payment on the large project happens in parallel with working on the small project (and thus is not on the critical path).

First, a correction: If there are two small projects, then both have payments expedited and the sequence does not matter. If there are two large projects, then neither has payments expedited and the sequence does not matter.

@JNing0 in this sense, there is a difference between large and small projects.

Second, a new theory: But we are not observing the entire portfolio of projects of the contractor. Probably, the more project we observe, the more projects the contractor has overall, not only with the government. Those outside projects are not affected by the QuickPay. They will be expedited at the expense of government small projects. Thus, as Jie suggested, if the number of projects we observe is higher, there are more opportunities to choose a project to prioritize.

[vibhuti6]

Thank you both for your helpful comments!

• The results are similar to the existing portfolio theory (here) when we use number of large projects as the metric. There's no effect in the full sample, but we see significant positive effect on the restricted sample -- please see the tables below. I find "proportion of large projects" to be a better metric though as it is normalized (so avoids suspicion that results are driven by outliers).

• As I understand, the new theory also relies on the assumption of multiple projects with different payment times. But we have no way to verify whether the contractor holds "private projects" and if so, what are their payment terms. This may be too tenuous of a claim to make empirically. Since the crux of the argument is the same between the two theories, my preference is to go with our original version.

Full sample

Restricted sample

[harishk05]

Thank you all for the comments and results.

[JNing0]

Thank you, Vibhuti!

One question is why the result is significant only in the restricted sample when we use large projects, i.e., on contractors that are active before QP? Does the theory distinguish between contractors who are active and after QP? I don't think so... Outliers under the absolute number of large projects is a valid concern. But it does not seem to be an issue: we get similar results using the number or the budget fraction: null result in the full sample and significant result in the restricted sample.

The result with the number of all projects, on the other hand, seems to be more robust. We have significant results both in the full and restricted sample. Could you do the quintile analysis using the number of all projects? Thanks!

[vibhuti6]

Hi Jie, the restricted sample in Table 12 is the one where contractors hold at least one large and one small project. That is, contractors holding only large or only small projects are excluded. This ensures there is at least some payment disparity between the portfolio of projects that a contractor holds.

I had run the regressions in Tables 5 and 6 with the congestion theory in mind. I had restricted the sample to contractors active before Quickpay to see whether existing contractors behave differently from new entrants in terms of the number of projects they take on. The figure in the corresponding comment (replicated below) shows the results for quintile analysis.

[JNing0]

Thank you for the clarification!

[JNing0]

I still find the results using "large project" unconvincing. The same question remains, why we only see effect in the restricted sample? One possibility is as follows. In the regression, we are essentially fitting a linear model on the treatment effect, which I'll write schematically as "b0 + b1*Num Large Projects". In the full sample, the intercept b0 is roughly estimated using small projects that belong to contractors that have small projects only. And b1 roughly comes from small projects with co-existing large projects. If the effect of the latter is not significantly different from the baseline variability in b0, we get no statistical significance for b1. In the restricted sample, we delete all these baseline observations for b0. So we pick up some significance in the slope b1.

The same idea applies to the regression using budget fraction. That's why we get similar results using these two metrics.

[vibhuti6]

Hi everyone, a quick follow up on our conversation earlier. Tables 11 and 12 in this issue here already examine how treatment effect varies with "number of large projects" -- both on the full and restricted sample. We see an effect in the restricted sample, but not in the full one.

So all that's left to do now is run these same regressions but use "total number of projects" instead, correct? Just wanted to confirm. Please let me know if I am missing something. Thanks!

[JNing0]

Yes, that is correct. Since the "large project" specification does not work in the full sample, we only need to run the regression with total number of projects on the contractors that has at least one large project. I don't think we need to require contractors to have both small and large. Thanks!

[vibhuti6]

Thanks, Jie -- what would be the argument for "shuffling" if a contractor holds only large projects (i.e., when there's no small project receiving expedited payment)?

[JNing0]

There will be no shuffling but such projects will be in the control group anyways. And we are interested in how the small projects are affected.

I guess you are thinking along the lines of comparing large projects with and without co-existing small projects? According to our theory, the ones without small projects will not be affected, compare with those with small projects. But that would be a different regression, where the treated would be large projects that have co-existing small projects?

[vibhuti6]

Thank you, Jie -- I need to think more about this to better understand the question/model in my mind. I will follow up when I have more clarity. As such, please see below the results for two cases: (1) sample restricted to at least one large project, and (2) sample restricted to at least one small and one large project.

Case 1

Case 2

[JNing0]

Thank you, Vibhuti. The results look good.

What follows is something we can further explore. Let's divide the large projects into two groups.

• Group A: a large project that belongs to a contractor that does not have small projects and thus is unaffected by QP.
• Group B: a large project that belongs to a contractor that has at least one small project.

A likely outcome of prioritizing large projects is that, compared to group A projects, project B projects would be expedited due to QP. In other words, group B projects are indirectly treated by QP via its effect on the co-existing small projects. There is a spill-over effect.

To test this, we can do a DiD on large projects. Group A would be the control group and group B the treatment group. Specifically, we can use the number of co-existing small projects as a proxy for treatment intensity. This will give us an intensity model for DiD.

The hypothesis would be that the treatment effect is negative, i.e., the treated large projects are expedited. The more co-existing small projects, the more significant the expedition.

[vibhuti6]

Hi everyone, here are the results based our discussion yesterday (code here). Please review and comment. Thanks!

DID on large projects (as outlined by Jie above)

Hypothesis: Negative treatment effect. Large projects whose contractors hold small projects will be expedited.

DID on small projects (with clean control group)

• Treatment group: Treat equals one for small projects with at least one large project in the same quarter.
• Control group: Treat is zero for large projects with NO small project in the same quarter.
• Excluded observations: Treat is not defined for other cases i.e, only small projects or large projects with small projects are excluded.
• Hypothesis: Positive treatment effect. Small projects whose contractors hold large projects are delayed, control group is large projects held by contractors with only large projects.

Other theories

I also re-ran other regressions by restricting the sample to contractors that have only one type of project. The results are robust for baseline, stage theory, and competition theory. We lose statistical significance for financial constraints theory with this restriction, but the sign is still positive.

[JNing0]

Thank you, Vibhuti! The results look great.

For the large project DiD, please also do a DiD with treatment intensity. The treatment intensity would be captured by the number of small projects owned by the same contractor. So in the interaction term, use NumOfSmallProjects x Post instead of Treat x Post. This allows us to see whether the prioritization effect increases with the number of co-existing small projects.

Jie Ning //////////////////////// Associate Professor Department of Operations Weatherhead School of Management Case Western Reserve University Cleveland, OH 44106 e-mail: @.*** tel: 216-368-3841 ////////////////////////

On Fri, Aug 5, 2022 at 4:46 PM Vibhuti Dhingra @.***> wrote:

Hi everyone, here are the results based our discussion https://github.com/QuickPay-Operational-Performance/Data-and-code/blob/master/notes/portfolio_regressions_notes_04-08-2022.pdf yesterday (code here https://github.com/QuickPay-Operational-Performance/Data-and-code/blob/master/qp_first_implementation/qp_first_pc_delay-2.Rmd). Please review and comment. Thanks! DID on large projects (as outlined by Jie above)

Hypothesis: Negative treatment effect. Large projects whose contractors hold small projects will be expedited.

[image: Screen Shot 2022-08-05 at 4 34 09 PM] https://user-images.githubusercontent.com/47764977/183158623-65d560ea-f494-41e4-b362-61fb6b3a6f39.png DID on small projects (with clean control group)

• Treatment group: Treat equals one for small projects with at least one large project in the same quarter.
• Control group: Treat is zero for large projects with NO small project in the same quarter.
• Excluded observations: Treat is not defined for other cases i.e, only small projects or large projects with small projects are excluded.
• Hypothesis: Positive treatment effect. Small projects whose contractors hold large projects are delayed, control group is large projects held by contractors with only large projects.

[image: Screen Shot 2022-08-05 at 4 38 17 PM] https://user-images.githubusercontent.com/47764977/183159107-7b45cca2-f806-421f-b22d-bfa20fedf098.png Other theories

I also re-ran other regressions by restricting the sample to contractors that have only one type of project. The results are robust for baseline, stage theory, and competition theory. We lose statistical significance for financial constraints theory with this restriction, but the sign is still positive.

— Reply to this email directly, view it on GitHub https://github.com/QuickPay-Operational-Performance/Data-and-code/issues/85#issuecomment-1206851670, or unsubscribe https://github.com/notifications/unsubscribe-auth/AOBDD5QPM4FXZXZ4AEYPF3DVXV4SFANCNFSM5267QNLQ . You are receiving this because you were mentioned.Message ID: <QuickPay-Operational-Performance/Data-and-code/issues/85/1206851670@ github.com>

[vibhuti6]

Thanks, Jie -- the effect does not seem to increase with number of small projects, please see below.

[JNing0]

Thanks. This is something that we can mention in the paper, i.e., the number of small projects does not seem to affect the magnitude of expedition for large projects. It is more like an "on-off" mechanism.

For the DiD result on small projects, we also need to rerun the regression with three-way interaction terms as in here. We can try both the "total number of projects" and the "number of large projects" per contractor as the mediator.

[vibhuti6]

Hi Jie, there's not much of an effect with either of these metrics:

[JNing0]

So the number of projects does not seem to affect the effect. Let's try a binary approach then.

Define indicator LB_i = 1 if small project i has co-existing large projects (from the same contractor) at any point of our observation horizon and 0 otherwise. So LB_i is analogous to the contract financing indicator CF. We can run a DiD regression similar to that with CF, with the control group being large projects without coexisting small projects. Then the triple-interaction term Treat x Post x LB gives us the additional treatment effect due to large projects. The coefficient should be positive.

[JNing0]

Correction: The LB indicator should be LB_i = 1 if small project i has co-existing large projects (from the same contractor) after QP launch and 0 otherwise.

[vibhuti6]

Hi everyone, here are some updates on the regression we discussed last time (also outlined by Jie in the comments above). Please review and comment. Thanks!

Model 1

• Suppose LB_i = 1 if small project i has co-existing large projects in a given time period. In this case, we cannot tease out the marginal effect using contract financing model since LB_i=1 only for small projects.
• This is because LB_i = 1 will imply that Treat_i = 1. Accordingly, LB_i x Post_t = 1 will imply that LB_i x Treat_i x Post_t = 1. Similarly, LB_i = 1 implies that LB_i x Treat_i = 1.
• In other words, some of the terms we used in contract financing regression become redundant here, and we get an error if we try to run the full specification. Relatedly, we get the same regression coefficient whether we use LB_i x Treat_i x Post_t or LB_i x Post_t.
• I have included below the results where redundant coefficients are removed. In this table, Treat_{i,l} is the same as LB_i.

Model 2

• A more appropriate model may be to consider whether the contractor has a concurrent large project in a given time period, regardless of whether the focal project is large or small.
• There is no collinearity with this specification, because the variable is defined independent of Treat_i
• We do not find any positive effect with this scenario, the sign is in fact negative and significant is most specifications. I am not sure how to interpret it at this point.

Misc. notes

• In both the above models, I removed observations where large projects co-exist with small ones. That is, we have a clean control group.
• The code for the regressions is here for reference.

[JNing0]

Thanks, Vibhuti.

Model 1 looks good and agrees with our expectation. Small projects with concurrent large projects experience an additional delay under QP. This additional delay is predicted by the portfolio theory.

I don't understand Model 2. So in the control group, all projects have that indicator equal one, except when the contractor owns no other large projects?

I sort of see where that model comes from, to get a regression parallel to the contract financing model. But that is not our goal. CF model is not THE model for such analysis. In what we have here, the treatment group has two subgroups but not the control group. That is perfectly OK. Model 1 is the correct model for our purpose: to check whether the two subgroups respond differently to the treatment.

[vibhuti6]

Thanks, Jie -- I think I understand my confusion. Model 1 does give us the additional effect on small projects with concurrent large projects, as predicted by the portfolio theory. My concern is simply that the regression variables are not independent (LB_i = 1 => Treat_i = 1), but not sure if there's a way around it given the setting.