Compatibility with Gurobi as alternative to CPLEX

[pblaney]

Hi @mskilab,

I've been pretty excited to get this tool in place for some while. After some issues with trying to get ahold of IBM to register for an academic license, I went with Gurobi as suggested. The process was fairly simple with a little adjustment to some misdirected dynamic library paths to get JaBbA to load the Gurobi library properly.

However, after a few test runs, I've narrowed in on some roadblocks in completing the run with Gurobi. It seems the error stems from expectations of using CPLEX at a few key points.

Below is a snippet of the error, I attached the log file with the complete output for reference:

JaBbA 2023-02-03 16:45:57: Number of gaps with nonzero width: 17908
JaBbA 2023-02-03 16:45:57: Number of segments before gap filtering: 25766
JaBbA 2023-02-03 16:45:58: 11200 segments produced after gap filtering
JaBbA 2023-02-03 16:45:58: creating karyograph
JaBbA 2023-02-03 16:46:27: loaded hets
JaBbA 2023-02-03 16:47:11: Definining coverage good quality nodes as >=50% bases covered by non-NA and non-Inf values in +/-100KB region
JaBbA 2023-02-03 16:47:11: Hard setting 0 Mb of the genome to NA that didn't pass our quality threshold
JaBbA 2023-02-03 16:47:11: Using loess to fit mean to variance relationship in segments with greater than 1 bins
JaBbA 2023-02-03 16:47:12: Using Sequenza to estimate purity ploidy
JaBbA 2023-02-03 16:47:17: Starting BAF model fit
JaBbA 2023-02-03 17:04:50: Built gGraph with 22712 nodes, 16136 edges, purity 0.97, and ploidy 4.4
JaBbA 2023-02-03 17:05:18: Brand new function for reciprocal junctions calling.
.
JaBbA 2023-02-03 17:05:18: Excluding 0 aberrant junctions whose both breakpoints are in NA coverage regions
JaBbA 2023-02-03 17:05:18: Cancel nudge for 4 aberrant junctions where one of the 2 breakpoint is in NA coverage regions
JaBbA 2023-02-03 17:05:18: In sum, we are forcing 0 junctions, excluding 0 junctions, and nudging 5 junctions
JaBbA 2023-02-03 17:05:18: number of specified max.threads: 12
JaBbA 2023-02-03 17:05:18: number of specified max.threads after processing: 12
sh: /Applications/CPLEX_Studio_Community2211//cplex/bin/arm64_osx/cplex: Bad CPU type in executable
Error in if (vnums[1] <= 12 && vnums[2] <= 6) { : 
  missing value where TRUE/FALSE needed
Calls: suppressPackageStartupMessages ... JaBbA -> jabba_stub -> ramip_stub -> .cplex_customparams
In addition: There were 24 warnings (use warnings() to see them)
Execution halted

As you can see, JaBbA makes a call/reference to my installation of CPLEX despite my selection of using Gurobi. Looking into this a bit deeper, I can see it's related to the function .cplex_customparams(). Within in this function, there is a parameter --customparams which evokes the following conditional:

    if (customparams)
    {
        if (verbose) jmessage("number of specified max.threads: ", max.threads)
        MAX.THREADS = Sys.getenv("LSB_DJOB_NUMPROC")
        if (nchar(MAX.THREADS) == 0)
            MAX.THREADS = Inf
        else
            MAX.THREADS = as.numeric(MAX.THREADS)
        max.threads = min(max.threads, MAX.THREADS)
        if (is.infinite(max.threads))
            max.threads = 0
        if (verbose) jmessage("number of specified max.threads after processing: ", max.threads)

        param.file = paste(out.file, '.prm', sep = '')
        .cplex_customparams(param.file, max.threads, treememlim = mem * 1024,
                            workingmemlim = floor((mem * 0.6) * 1024))
        if (verbose) {
            jmessage("param.file contents: ")
            system2('cat', normalizePath(param.file))
        }

        Sys.setenv(ILOG_CPLEX_PARAMETER_FILE = normalizePath(param.file))
        if (verbose)
        {
            jmessage('Creating ILOG CPLEX PARAMETER FILE in ', Sys.getenv('ILOG_CPLEX_PARAMETER_FILE'))
        }
    }

Now, there is no issue with that conditional. The problem is this parameter customparams is always set to TRUE when propagated from the parent function ramip_stub() at lines 1261-1288:

ramip_stub(kag.file,
                   jabba.raw.rds.file,
                   mc.cores = mc.cores,
                   max.threads = max.threads,
                   mem = max.mem,
                   tilim = tilim,
                   edge.nudge = edgenudge,
                   use.gurobi = use.gurobi,
                   ab.force = ab.force,
                   ab.exclude = ab.exclude, ## we now exclude things during karyograph_stub
                   ## ab.exclude = integer(0),
                   init = init,
                   verbose = verbose,
                   purity.min = purity,
                   mipstart = mipstart,
                   epgap = epgap,
                   purity.max = purity,
                   ploidy.min = ploidy,
                   ploidy.max = ploidy,
                   slack.prior = 1/slack.penalty,
                   loose.penalty.mode = loose.penalty.mode,
                   dyn.tuning = dyn.tuning,
                   lp = lp,
                   ism = ism,
                   tfield = tfield,
                   fix.thres = fix.thres,
                   min.bins = min.bins,
                   customparams = T)

I think this could be easily adjusted given that it seems the function of the customparams parameter is directly related to CPLEX users. Therefore, you could perhaps set customparams = !use.gurobi. Also, maybe it is feasible to create an alternative process for similar resource settings with Gurobi.

Let me know if I can provide more info to help troubleshoot. I may cut a branch and test this in the meantime. Best, Patrick jabba.log

[zining01]

Hi again!

It looks like you're right - if you haven't tried this modification already, could you please try reinstalling from the following branch to see if it fixes the issue? https://github.com/mskilab/JaBbA/tree/zc_dev

There are some other minor bug fixes in this branch but hopefully they won't affect your analysis. Please let me know how it goes!

[pblaney]

Hello Zi-Ning,

After pulling the updated dev branch, I can confirm JaBbA executed without an issue. Based on the log, the Gurobi Optimizer was invoked correctly.

.....
Starting optimization with gurobi!
Gurobi Optimizer version 10.0.0 build v10.0.0rc2 (mac64[x86])

CPU model: Intel(R) Core(TM) i7-8569U CPU @ 2.80GHz
Thread count: 4 physical cores, 8 logical processors, using up to 8 threads

I will notify you if I have any further issues but otherwise this issue has been resolved. Thanks again, Patrick