CPLEX output goes to stdout/stderr

[svigerske]

I updated to the most recent version and now get CPLEX output on stdout/stderr, even though I pass my own output sink when constructing SHOT::Solver.

Could be that I just didn't notice this before because CPLEX output was always disabled, but this one in MIPSolverCplex::checkParameters() is new:

    if(env->reformulatedProblem->properties.numberOfNonlinearConstraints == 0)
        env->settings->updateSetting("Console.DualSolver.Show", "Output", true);

?

I haven't checked with Gurobi and Cbc.

[andreaslundell]

I removed those exact lines today actually. But I think the reason for the output is intentional, as I now decided to print out the MIP solver output if the problem (after reformulation) is a problem that is solved by the MIP solver in one iteration (MILP, MIQP, MIQCQP,...). The motivation is that the reformulated problem might take a long time to solve without any output and the user would not have any idea of the progress.

What problem are you trying to solve and what is the exact output?

[svigerske]

I was running MIQCP01 from GAMS testlib. I guess it's a convex MIQCP and you pass this directly to CPLEX.

My problem is not that there is CPLEX output, but that it isn't redirected to the output stream that I gave to SHOT.

[andreaslundell]

No, I think the MIP solver output goes directly to stdout/stderr (and has always done so). I have not checked if it is possible to redirect it.

[svigerske]

I'm sure it is.

But if it's off by default again anyway, it's less critical to me.

[svigerske]

Hmm, no, it's not off. You enable it in Solver.cpp now instead.

[andreaslundell]

Yes, I just moved it here https://github.com/coin-or/SHOT/blob/6e1409bfc356cc6f51c05bd380a04a3ae5290f14/src/Solver.cpp#L477

As a quick fix, I guess we could set Output.Console.DualSolver.Show=false when calling from GAMS... It seems to be possible to redirect Cplex output to a stream, but I have not found any similar functionality in Gurobi.

[svigerske]

Hmm, maybe.

For Gurobi (C API), the general callback can be used, e.g.,

static int __stdcall grbcallback(GRBmodel* model, void* qcbdata, int where, void* usrdata)
{
   if( where == GRB_CB_MESSAGE )
   {
      char* msg;
      GRBcbget(qcbdata, where, GRB_CB_MSG_STRING, &msg);
      gevLogPChar((gevHandle_t)usrdata, msg);
   }

   if( gevTerminateGet((gevHandle_t)usrdata) )
      GRBterminate(model);

   return 0;
}

Set via GRBsetcallbackfunc.

For Cbc you can create you own message handler class (e.g., https://github.com/coin-or/GAMSlinks/blob/master/src/cbc/GamsCbc.cpp#L57) and set it via model->passInMessageHandler(msghandler);

[svigerske]

I tried adding

env->settings->updateSetting("Console.DualSolver.Show", "Output", false);

in EntryPointsGAMS.cpp, but there is no proper place to do this.

If I set the option before my call to solver.setProblem(problem, modelingSystem), the latter will call Solver::selectStrategy(), which overwrite my setting. And then it seems to go to MIPSolverCplex::initializeSolverSettings() almost immediately. So setting the option after solver.setProblem(problem, modelingSystem) has no effect anymore.

It's generally a bad idea if a solver overwrites user options.

[andreaslundell]

Yeah, I will add the callbacks required to the MIP solvers since I do not think it is a good idea to remove the option to print out the MIP solution output for GAMS users...

[andreaslundell]

I did this for Gurobi and Cbc in https://github.com/coin-or/SHOT/tree/redirect_output.

Did you have example code for Cplex as well? :-)

[andreaslundell]

I could not seem redirect the message "Academic license - for non-commercial use only" in Gurobi though, but does not know if this matters to you..?

[svigerske]

I would have for the C interface of CPLEX.

For C++, I think you can pass your own ostream& around the place where you have

            cplexInstance.setOut(cplexEnv.getNullStream());

https://www.ibm.com/support/knowledgecenter/SSSA5P_12.10.0/ilog.odms.cplex.help/refcppcplex/html/classes/IloAlgorithm.html#method_setOut

Maybe spdlog provides an ostream adapater for an output-sink?

[svigerske]

But that's what I use when I need an ostream that prints to the GAMS log:

/** streambuf implementation that directs output to gev streams */
/* #define BUFFERSIZE GMS_SSSIZE */
#define BUFFERSIZE 1
class GamsOutputStream : public std::streambuf
{
private:
   gevHandle_t gev;
   char buffer[BUFFERSIZE+2];
   bool tostat;

public:
   GamsOutputStream(
      gevHandle_t gev_,
      bool        tostat_
   )
   : gev(gev_), tostat(tostat_)
   {
      setp(buffer, buffer+BUFFERSIZE);
   }

   ~GamsOutputStream()
   {
      overflow(traits_type::eof());
   }

   int_type overflow(int_type c = traits_type::eof())
   {
      if( c != traits_type::eof() )
      {
         *pptr() = traits_type::to_char_type(c);
         *(pptr()+1) = '\0';
      }
      else
         *pptr() = '\0';

      if( tostat )
         gevLogStatPChar(gev, pbase());
      else
         gevLogPChar(gev, pbase());

      setp(buffer, buffer+BUFFERSIZE);

      return 0;
   }
};

[svigerske]

I could not seem redirect the message "Academic license - for non-commercial use only" in Gurobi though, but does not know if this matters to you..?

Doesn't really matter.

Maybe you can get rid of it by moving the line

 gurobiModel->getEnv().set(GRB_IntParam_LogToConsole, 0);

in between

        gurobiEnv = std::make_shared<GRBEnv>();
        gurobiModel = std::make_shared<GRBModel>(*gurobiEnv.get());

in MIPSolverGurobi::initializeProblem()?

[andreaslundell]

I could not seem redirect the message "Academic license - for non-commercial use only" in Gurobi though, but does not know if this matters to you..?

Doesn't really matter.

Maybe you can get rid of it by moving the line

 gurobiModel->getEnv().set(GRB_IntParam_LogToConsole, 0);

in between

        gurobiEnv = std::make_shared<GRBEnv>();
        gurobiModel = std::make_shared<GRBModel>(*gurobiEnv.get());

in MIPSolverGurobi::initializeProblem()?

Nope, the message seems to be done in the constructor of the environment somehow.

[svigerske]

Sounds like a Gurobi bug then (well, not a critical one). It also somehow feels familiar. There was a Gurobi message when one has a beta version that was difficult to suppress.