search

jkcshea / ivmte

An R package for implementing the method in Mogstad, Santos, and Torgovitsky (2018, Econometrica).
GNU General Public License v3.0
18 stars 2 forks source link

Audits that don't end #199

Closed jkcshea closed 2 years ago

jkcshea commented 3 years ago

I've found that some audits never end. This is because gurobi is violating its feasibility tolerance.

For example, the default settings are audit.tol = 1e-06 for ivmte, and FeasibilityTol = 1e06 for Gurobi. Yet, the audit will get stuck in a loop where it repeatedly violates the same points in the audit grid. Reducing FeasibilityTol has no effect.

So I plan to incorporate a check for whether the audit is stuck in a loop. If so, then a warning will be returned, and the audit.tol will be increased.

a-torgovitsky commented 3 years ago

I wonder if it makes more sense to just use Gurobi to check feasibility? (This can be done by setting the objective function to 1 I think. Gurobi should have special algorithms to handle this case.) That way there aren't two tolerances floating around. Question is whether we take a huge performance hit for this.

jkcshea commented 3 years ago

Sure, I can try this out.

One way to do it is to pass the entire audit grid to gurobi, but from what I recall, this can become very memory intensive. I remember John and Christine were able to generate memory errors using MTR specifications with many knots. (That may have been the reason why I switched to sparse matrices whenever possible.)

So I think the better way is to continue doing what we do now, and test each type of shape constraint separately. Except we'll now have gurobi check feasibility for us.

a-torgovitsky commented 3 years ago

That makes sense as a solution. I guess whether it works will depend on how much overhead is involved for Gurobi to do these feasibility checks. Let's make sure it's not going to be a computational drag before going ahead with it.

If it is a computational drag then maybe your proposed tolerance adjusting procedure is better. Just my experience with putting in those types of ad hoc fixes is that it's hard to imagine all of the possible ways in which it can potentially fail. And you end up continually modifying it until you have a real Rube Goldberg-esque solution.

a-torgovitsky commented 2 years ago

Can we close this now @jkcshea ?

jkcshea commented 2 years ago

Oh, yes we can.