Advice on dealing with numerical issues

[cblandhol]

@johnnybonney and I have been re-running the results for our Angrist and Evans application with the direct MTR approach and are running into the "numerical issues" error mentioned in issue #186. Do you have any advice on how to deal with this issue?

Below I have made a reproducible example which illustrates the issue with the same specification and similar number of covariate values we use in our application, but which uses the dataset provided in the package.

library(ivmte)
library(data.table)
library(gurobi)

data(AE)
data <- as.data.table(AE)

# rename variables
data[, instrument := samesex]
data[,  y := worked]
data[ , treatment:= morekids]

# covariate in our application takes values between 1 and 15
data[ , x := yob - 43]
x.max <- max(data$x)

# estimate propensity score
sat.pscore       <- as.formula(paste0("treatment ~ factor(instrument)*factor(x)"))

sat.pscore.res   <<- glm(formula = sat.pscore, 
                        data = data, 
                        family = "binomial")

data$pscore      <- predict(sat.pscore.res, type = "response")

# now, constant spline with knots at propensity scores for each value of x
degree = 0
x_knots  <- paste0(sort(unique(data[data$x == 1]$pscore)), collapse = ",")
u_part   <- paste0("~ factor(x) + as.integer(x == ",1, "):uSplines(knots =c(", x_knots,"), degree = ", degree, ")")

for (i in 2:x.max) {
    x_knots <- paste0(sort(unique(data[data$x == i]$pscore)), collapse = ",")
    u_part   <- paste0(u_part," + ", "as.integer(x == ",i, "):uSplines(knots = c(", x_knots,"), degree = ", degree, ")")
}
mtr.spec <- u_part
mtr.spec <- as.formula(mtr.spec)

# collect arguments
args <- list(
    data   = data,
    outcome = "y",
    m1 = mtr.spec,
    m0 = mtr.spec,
    propensity = treatment ~ factor(instrument)*factor(x),
    noisy = T,
    initgrid.nx = 15,
    audit.nx = 15,
    audit.max = 10,
    audit.nu = 40000,
    initgrid.nu = 10000,
    solver = "gurobi",
    target = "ate"
)

# Direct MTR approach
args$criterion.tol <- 0
res  <- try(do.call(ivmte, args))

Solver: Gurobi ('gurobi')

Obtaining propensity scores...

Generating target moments...
    Integrating terms for control group...
    Integrating terms for treated group...

Performing direct MTR regression...
    MTR is not point identified.

Performing audit procedure...
    Generating initial constraint grid...

    Audit count: 1
    Minimum criterion: 0.2420845
Warning:  The solver was unable to satisfy the optimality tolerance when minimizing the criterion, so a suboptimal solution is returned. Tolerance parameters for the solver can be passed through the argument 'solver.options'. 
    Obtaining bounds...
    Model resulted in numerical issues.

Setting presolve = FALSE results in an uninformative error message:

args$solver.presolve <- FALSE
res  <- try(do.call(ivmte, args))

Solver: Gurobi ('gurobi')

Obtaining propensity scores...

Generating target moments...
    Integrating terms for control group...
    Integrating terms for treated group...

Performing direct MTR regression...
    MTR is not point identified.

Performing audit procedure...
    Generating initial constraint grid...

    Audit count: 1
    Minimum criterion: 0.2421279
    Obtaining bounds...
    Model was unbounded.

    Restarting audit with new settings:
    initgrid.nx =  15 
    initgrid.nu =  15000 
    Generating initial constraint grid...

    Audit count: 1
    Minimum criterion: 0.2421279
    Obtaining bounds...
    Model was unbounded.

    Restarting audit with new settings:
    initgrid.nx =  15 
    initgrid.nu =  22500 
    Generating initial constraint grid...

    Audit count: 1
    Minimum criterion: 0.2421279
    Obtaining bounds...
    Model was unbounded.

    Restarting audit with new settings:
    initgrid.nx =  15 
    initgrid.nu =  33750 
    Generating initial constraint grid...

    Audit count: 1
    Minimum criterion: 0.2421279
    Obtaining bounds...
Error in abs(x) : non-numeric argument to mathematical function

The MST estimator does not return the same issue:

# MST estimator
ivlike      <- c(y ~ treatment*factor(instrument)*factor(x))
components  <- l()
args$ivlike     <- ivlike
args$components <- components
res  <- try(do.call(ivmte, args))

Obtaining propensity scores...

Generating target moments...
    Integrating terms for control group...
    Integrating terms for treated group...

Generating IV-like moments...
    Moment 1...
    Moment 2...
    Moment 3...
    Moment 4...
    Moment 5...
    Moment 6...
    Moment 7...
    Moment 8...
    Moment 9...
    Moment 10...
    Moment 11...
    Moment 12...
    Moment 13...
    Moment 14...
    Moment 15...
    Moment 16...
    Moment 17...
    Moment 18...
    Moment 19...
    Moment 20...
    Moment 21...
    Moment 22...
    Moment 23...
    Moment 24...
    Moment 25...
    Moment 26...
    Moment 27...
    Moment 28...
    Moment 29...
    Moment 30...
    Moment 31...
    Moment 32...
    Moment 33...
    Moment 34...
    Moment 35...
    Moment 36...
    Moment 37...
    Moment 38...
    Moment 39...
    Moment 40...
    Moment 41...
    Moment 42...
    Moment 43...
    Moment 44...
    Moment 45...
    Moment 46...
    Moment 47...
    Moment 48...
    Moment 49...
    Moment 50...
    Moment 51...
    Moment 52...
    Moment 53...
    Moment 54...
    Moment 55...
    Moment 56...
    Moment 57...
    Moment 58...
    Moment 59...
    Independent moments: 59 

Performing audit procedure...
    Generating initial constraint grid...

    Audit count: 1
    Minimum criterion: 1.2216
    Obtaining bounds...
    Violations: 0
    Audit finished.

Bounds on the target parameter: [-0.6235989, 0.3173004]

[jkcshea]

Sounds good. The two methods are on separate branches that will most likely conflict if I try to merge them. So I will work on combining the two approaches.

@johnnybonney and @cblandhol, the Cholesky approach is on the branch bugfix/issue209-cholesky. Once I combine the two branches together, the only change you should need to make to your code are the ones @a-torgovitsky mentioned in the previous post.

[jkcshea]

Oops, I didn't mean to close this, but I suppose the issue is resolved. @cblandhol @johnnybonney The updates suggested by @a-torgovitsky have been implemented and merged into the master branch.