User-defined operator JuMP: UndefVarError: `∇f` not defined

[franckgaga]

I am opening a new issue since it may be related to another problem.

First, @frapac thanks for your work!

The current MadNLP.jl dev version does solve the want_hess issue indeed, on the MWE of the last issue. It may or may not be related, but on the specific case of ModelPredictiveControl.jl (that internally uses user-registered nonlinear operator):

using ModelPredictiveControl, JuMP, MadNLP
f(x,u,_) = 0.1x
h(x,_) = x
model = NonLinModel(f, h, 1, 1, 1, 1, solver=nothing)
nmpc = NonLinMPC(model, Hp=10, optim=Model(MadNLP.Optimizer))
nmpc([1])

there is no longer the want_hess error, but a new one:

ERROR: UndefVarError: `∇f` not defined
Stacktrace:
  [1] eval_constraint_jacobian_transpose_product(block::MadNLPMOI.QPBlockData{…}, Jtv::Vector{…}, x::Vector{…}, v::Vector{…})
    @ MadNLPMOI ~/.julia/dev/MadNLP/ext/MadNLPMOI/utils.jl:528
  [2] eval_constraint_jacobian_transpose_product(model::MadNLPMOI.Optimizer, Jtv::Vector{…}, x::Vector{…}, v::Vector{…})
    @ MadNLPMOI ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:768
  [3] jtprod!
    @ ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:810 [inlined]
  [4] _eval_jtprod_wrapper!(cb::MadNLP.SparseCallback{…}, x::Vector{…}, v::Vector{…}, jvt::Vector{…})
    @ MadNLP ~/.julia/dev/MadNLP/src/nlpmodels.jl:620
  [5] eval_lag_hess_wrapper!(solver::MadNLPSolver{…}, kkt::MadNLP.SparseKKTSystem{…}, x::MadNLP.PrimalVector{…}, l::Vector{…}; is_resto::Bool)
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/callbacks.jl:169
  [6] eval_lag_hess_wrapper!
    @ ~/.julia/dev/MadNLP/src/IPM/callbacks.jl:142 [inlined]
  [7] regular!(solver::MadNLPSolver{…})
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/solver.jl:262
  [8] solve!(nlp::MadNLPMOI.MOIModel{…}, solver::MadNLPSolver{…}, stats::MadNLP.MadNLPExecutionStats{…}; x::Nothing, y::Nothing, zl::Nothing, zu::Nothing, kwargs::@Kwargs{})
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/solver.jl:165
  [9] solve!
    @ ~/.julia/dev/MadNLP/src/IPM/solver.jl:128 [inlined]
 [10] solve!
    @ ~/.julia/dev/MadNLP/src/IPM/solver.jl:14 [inlined]
 [11] solve!(solver::MadNLPSolver{…})
    @ MadNLP ~/.julia/dev/MadNLP/src/IPM/solver.jl:17
 [12] optimize!(model::MadNLPMOI.Optimizer)
    @ MadNLPMOI ~/.julia/dev/MadNLP/ext/MadNLPMOI/MadNLPMOI.jl:956
 [13] optimize!
    @ ~/.julia/packages/MathOptInterface/2rAFb/src/Bridges/bridge_optimizer.jl:380 [inlined]
 [14] optimize!
    @ ~/.julia/packages/MathOptInterface/2rAFb/src/MathOptInterface.jl:85 [inlined]
 [15] optimize!(m::MathOptInterface.Utilities.CachingOptimizer{…})
    @ MathOptInterface.Utilities ~/.julia/packages/MathOptInterface/2rAFb/src/Utilities/cachingoptimizer.jl:316
 [16] optimize!(model::Model; ignore_optimize_hook::Bool, _differentiation_backend::MathOptInterface.Nonlinear.SparseReverseMode, kwargs::@Kwargs{})
    @ JuMP ~/.julia/packages/JuMP/as6Ji/src/optimizer_interface.jl:457
 [17] optimize!
    @ ~/.julia/packages/JuMP/as6Ji/src/optimizer_interface.jl:409 [inlined]
 [18] optim_objective!(mpc::NonLinMPC{Float64, UnscentedKalmanFilter{…}, Model, ModelPredictiveControl.var"#98#100"})
    @ ModelPredictiveControl ~/.julia/dev/ModelPredictiveControl/src/controller/execute.jl:476
 [19] moveinput!(mpc::NonLinMPC{…}, ry::Vector{…}, d::Vector{…}; Dhat::Vector{…}, Rhaty::Vector{…}, Rhatu::Vector{…}, ym::Nothing, D̂::Vector{…}, R̂y::Vector{…}, R̂u::Vector{…})
    @ ModelPredictiveControl ~/.julia/dev/ModelPredictiveControl/src/controller/execute.jl:67
 [20] moveinput!(mpc::NonLinMPC{…}, ry::Vector{…}, d::Vector{…})
    @ ModelPredictiveControl ~/.julia/dev/ModelPredictiveControl/src/controller/execute.jl:52
 [21] #_#123
    @ ~/.julia/dev/ModelPredictiveControl/src/predictive_control.jl:49 [inlined]
 [22] PredictiveController
    @ ~/.julia/dev/ModelPredictiveControl/src/predictive_control.jl:44 [inlined]
 [23] (::NonLinMPC{Float64, UnscentedKalmanFilter{…}, Model, ModelPredictiveControl.var"#98#100"})(ry::Vector{Int64})
    @ ModelPredictiveControl ~/.julia/dev/ModelPredictiveControl/src/predictive_control.jl:44
 [24] top-level scope
    @ ~/Dropbox/Programmation/Julia/TestMPC/src/test_yo.jl:67
Some type information was truncated. Use `show(err)` to see complete types.

Originally posted by @franckgaga in https://github.com/MadNLP/MadNLP.jl/issues/318#issuecomment-2093398763

[frapac]

good catch @franckgaga , there was an issue with the MOI wrapper in MadNLP. The problem is that eval_constraint_jacobian_product is not used inside Ipopt, so it is less thoroughly tested than the other methods.

Your issue should be solved with #337 .

[franckgaga]

Just tried the branch and it does solve the issue, thanks!