Add `sense` support for all algorithms

[agerlach]

Currently sense is only supported for a subset of algorithms. This breaks the common interface as some solvers solve min vs max. Furthermore, since sense is a kwarg for OptimizationProblem and not solve, the interface and docs give the impression that it is supported for all algorithms.

using Optimization, OptimizationBBO

J(x,p) = x

F = OptimizationFunction(J)
prob = Optimization.OptimizationProblem(F, 0.0; lb = -10, ub =10, sense = MaxSense)
solve(prob, BBO_adaptive_de_rand_1_bin_radiuslimited()).objective # -10.0

[soldasim]

Also encountered this issue. The sense kwarg completely breaks some solvers but does so silently without any error.

Consider the following MWE:

using Optimization
using OptimizationOptimJL

obj(x, p) = x[1] + x[2]

lb, ub = [0.,0.], [10.,10.]
x0 = [5.,5.]

# without `sense`
prob = OptimizationProblem(OptimizationFunction(obj, AutoForwardDiff()), x0, nothing; lb, ub)
sol = solve(prob, LBFGS(); x_tol=0.1)
println("no-sense: $(sol.u)")

# with `MinSense`
prob = OptimizationProblem(OptimizationFunction(obj, AutoForwardDiff()), x0, nothing; lb, ub, sense=Optimization.MinSense)
sol = solve(prob, LBFGS(); x_tol=0.1)
println("MinSense: $(sol.u)")

# with `MaxSense`
prob = OptimizationProblem(OptimizationFunction(obj, AutoForwardDiff()), x0, nothing; lb, ub, sense=Optimization.MaxSense)
sol = solve(prob, LBFGS(); x_tol=0.1)
println("MaxSense: $(sol.u)")

This outputs:

julia> 

no-sense: [4.8342920314099165e-9, 4.8342920314099165e-9]
MinSense: [4.8342920314099165e-9, 4.8342920314099165e-9]
MaxSense: [5.0, 5.0]

As you can see the solver does nothing when MaxSense is provided.

[ChrisRackauckas]

@Vaibhavdixit02 can you prioritize adding a trait to catch this?

[Vaibhavdixit02]

What we'd like to do here is instead of changing it in the solvers handle it in the instantiate_function method and add the negative sign to the objective function evaluation and derivatives when MaxSense is passed