[Moco] IntermediateCallback Disabled

[tymo77]

Solving any sort of problem in Moco gives me the following warning before Ipopt starts iterating:

Warning: intermediate_callback is disfunctional in your installation. You will only be able to use stats(). See https://github.com/casadi/casadi/wiki/enableIpoptCallback to enable it.

I think this is a known issue, but I don't see an issue thread for it on here or the old Moco repository.

I get this error in all versions of Moco I have tried: the 0.4.0 release and when compiling opensim-core on Windows using superbuild.

While this does not, of course, affect the solution, I am quite interested in having intermediate iterates for the purposes of debugging and visualization of the solution process, so I am motivated to help resolve this. I haven't found the explanation in the link in the warning to be particularly helpful. I'm assuming that the inclusion of these header files shouldn't be an issue when building from source along with all the dependencies. Perhaps there is a workaround someone is using or a step I am missing?

[nickbianco]

Hi @tymo77. The intermediate callback is controlled via the output_interval property on MocoCasADiSolver. Have you tried enabling this property? I haven't tried using this feature recently, but it might work fine despite the warning you are seeing.

[tymo77]

If I set the output_interval, I get the following message with more information than the usual warning and the solver exists on iteration 0:

Warning: intermediate_callback is disfunctional in your installation. You will only be able to use stats(). See https://github.com/casadi/casadi/wiki/enableIpoptCallback to enable it. intermediate_callback: Error in Function::operator() for 'NlpsolCallback' [CallbackInternal] at C:\Users\tyler\opensim-core-4.2\dependencies\casadi\casadi\core\function.cpp:1368: C:\Users\tyler\opensim-core-4.2\dependencies\casadi\casadi\core\function_internal.cpp:3363: Failed to evaluate 'eval_dm' for NlpsolCallback: C:\Users\tyler\opensim-core-4.2\dependencies\casadi\casadi\core\callback_internal.cpp:122: Error calling "eval" for object NlpsolCallback: invalid type specifier

[nickbianco]

Looks like the current solution is to copy IPOPT headers into the include path manually: https://github.com/casadi/casadi/wiki/enableIpoptCallback.

[tymo77]

Thanks @nickbianco ! So I have really done my best to try and chase this one down on my own, but I can't seem to figure it out.

I wasn't sure which include path we need them in so I tried all the paths I could find that seemed relevant. I copied all of the Ipopt header files into each of the following directories directly and within a subdirectory labeled "\coin".

First, when building the dependencies:

• C:\Users\tyler\opensim-core\dependencies\casadi\external_packages\ipopt\3.10
• C:\Users\tyler\opensim-core\dependencies\casadi\external_packages\ipopt
• C:\Users\tyler\opensim-core\dependencies\casadi\casadi\interfaces\ipopt

And second within the dependency installations, when building OpenSim-Core:

• C:\Users\tyler\opensim-core-dependencies-install\casadi\casadi\include
• C:\Users\tyler\opensim-core-dependencies-install\ipopt\include

I rebuilt everything from scratch step-by-step but I still get the following warning message every time I run the MocoCasADiSolver:

Warning: intermediate_callback is disfunctional in your installation. You will only be able to use stats(). See https://github.com/casadi/casadi/wiki/enableIpoptCallback to enable it.

And the following errors when I actually set MocoCasADiSolver to output intermediate iterates:

intermediate_callback: Error in Function::operator() for 'NlpsolCallback' [CallbackInternal] at C:\Users\tyler\opensim-core\dependencies\casadi\casadi\core\function.cpp:1368: C:\Users\tyler\opensim-core\dependencies\casadi\casadi\core\function_internal.cpp:3363: Failed to evaluate 'eval_dm' for NlpsolCallback: C:\Users\tyler\opensim-core\dependencies\casadi\casadi\core\callback_internal.cpp:122: Error calling "eval" for object NlpsolCallback: invalid type specifier

Is there a different include path I should be using? Is there a change to a cmake file required?

What's interesting to me is that if I edit intermediateCallbackWithIterateImpl in MocoCasOCProblem.cpp to:

void intermediateCallbackWithIterateImpl(
            const CasOC::Iterate& iterate) const override {
        /*std::string filename =
                fmt::format("MocoCasADiSolver_{}_trajectory{:06i}.sto",
                        m_formattedTimeString, iterate.iteration);
        convertToMocoTrajectory(iterate).write(filename);*/
        std::cout << iterate.iteration << std::endl;
    }

It works as expected without any errors though it still shows the intermediate_callback disfunctional warning beforehand. I can set it to print the iterate number every N iterations, when I set_output_interval to N in the MocoCasADiSolver.

I can even bypass the error and write a trajectory file if I'd like, if I leave it like this:

void intermediateCallbackWithIterateImpl(
            const CasOC::Iterate& iterate) const override {
        /*std::string filename =
                fmt::format("MocoCasADiSolver_{}_trajectory{:06i}.sto",
                        m_formattedTimeString, iterate.iteration);*/
        convertToMocoTrajectory(iterate).write("asdf.sto");
        std::cout << iterate.iteration << std::endl;
    }

The trajectory file "asdf.sto" is the right format and size, but is only zeros, as if the iterate variables have been initialized but not updated.