Support for MathOptInterface.AbstractOptimizer that do not support MathOptInterface.NLPBlock

[ValentinKaisermayer]

There are two possibilities for this

• Use JuMP variable refs and put them through the generated objective and contraints functions. If they are affine and/or quadratic the result will be affine or quadratic expressions. This would introduce a dependency onto JuMP.
• Use the expressions directly and generate the affine and quadratic MOI terms directly from them.

I prefere option two.

@ChrisRackauckas Would it be ok to add Symbolics.jl as dependency for this? The other possibility is to generate the terms from the Julia expression. In both cases I would have to simplify and expand first, in order to be sure that it is really just an affine and/or quadratic expression. For a symbolic expression there is simplify. How could I achieve his from a Julia expression directly?

Another issue I see when using the Julia expressions in the problem, is that the parameters have already been replaced so no reinit! (as discussed in #352 ) would be possible. This is of course true for the current use case of the expressions as well.

[ChrisRackauckas]

Yes, adding Symbolics.jl for this is good. Hessian sparsity detection would tell you if it's linear, and then you calculate the Hessian to see if non-constant to get if it's quadratic. @shashi might have a better way via the tracer to make it return true/false for quadratic. In theory the algorithm would already know this because it replaces nonlinear expressions with polynomials, and if it ever does that if could flip a bool.

[ValentinKaisermayer]

it replaces nonlinear expressions with polynomials, and if it ever does that if could flip a bool.

This seems more like a job for MTK but would be very cool indeed and somewhat cutting edge.

[manuelbb-upb]

I recently thought about this, too, when discovering coeff and degree in Symbolics.

Coming from JuMP, I think there are some issues in Symbolics and SymbolicUtils with vector-algebra that we can keep in mind to make setting up linear or quadratic problems easy and avoid unexpected errors with scalarization or simplification. These are some of the pull-request I opened to (try to) address some of the issues I encountered when exploring Symbolics for this use-case:

• Ambiguity issues with dot-products, like #635 or #575. There is a clumsy pull-request where I try to solve the ambiguity issues automatically, but it's way over my head and might never work, so just defining the specializations for those two cases might be preferable in the near future.
• Setting up a linear objective via dot-product and calling scalarize.
• Quadratic objectives with integer exponent 2. Might require changes in SymbolicUtils, also see the follow-up.
• Simplification of linear or quadratic objectives involving broadcasts, like sum( x .* 2) or sum( x .^ 2).

As I am still relatively new to the whole ecosystem, some of the proposed changes might very-well be very dumb. I just wanted to link them because they popped up when thinking about Symbolics for LP and QP.

PS: if re-instantiation of an optimization problem with different parameters can be made to work, this would be great, because currently that's not well-supported in ParametricOptInterface, e.g., sum((param_matrix * variable_vector).^2) does not work there...

[ValentinKaisermayer]

The re-initalization will be handled via a caching interface, see #406, also for MOI. I started a PR for the MOI solver interface for LPs and QPs #381. It works via expressions and low-level MOI calls, but I feel it would be better to use MTK for simplification and checking if the problem is linear or quadratic and just using JuMP directly.

[manuelbb-upb]

I have just skimmed a few of the implementation in #381. Personally, I don't see the harm in using low-level MOI calls, but the Expr parsing indeed looks like something to outsource. Could the parsing and simplification be handled by Symbolics alone (avoiding MTK as dependency)? Then coeff and degree could perhaps be used to define the low-level MOI structures, as long as the parameter and constant metadata is there.

Please excuse, if these are silly questions, as I am not well versed in the internals of Optimization.jl and MTK :see_no_evil: What I originally thought about was using Symbolics as an alternative interface to MOI, independent even from Optimization.jl: Set up a caching MOI Optimizer that can utilize all the cool bridges for LP and QP, but generate the MOI functions with Symbolics instead of JuMP. Also borrow the MTK parameter metadata (separate mini-package?) to distinguish variables from parameters and then have some substitution mechanism for expressions like sum((param_matrix * variable_vector).^2) which don't yet work with ParametricOptInterface and JuMP. In the end, it was merely a thought experiment without pressing priority, but much of what is in #381 could be re-used for something like this.

[ValentinKaisermayer]

I have to say the idea is actually quite nice. JuMP lacks an interface for parameters. At least a real on, ParametricOptInterface can not handle more complex stuff. I have experimented with that already and I think it would be not difficult to make an interface. I.e. Given an OptimizationProblem, replace the parameters, analyse the expression, if linear or quadratic build a JuMP.AffExpr or JuMP.QuadExpr or if not a nonlinear one. Something like a constructor for a JuMP.Model from an OptimizationProblem.