Open cossio opened 5 years ago
Dear Cossio, Did you solve your problem? I'm interested in this issue as well. Do you use JuMP or NLopt directly? I would like to give gradients to my objective function using the JuMP interface language but I'm not able to do it. Do you know how to do it?
Thank you for your attention
Giada
@giadasp I did not figure out how to pass a preconditioner to NLopt. But Optim has a documented API for this purpose, https://github.com/JuliaNLSolvers/Optim.jl,
https://github.com/JuliaOpt/NLopt.jl/blob/6fd18707b633978bfb3070b8fa514adcd9fe5289/src/libnlopt.jl#L143-L149 but I don't have an example of exactly what this should take.
Here's an example:
using NLopt, Test
begin
function nlopt_scalar_callback(n, p_x, p_grad, f_data)
data = unsafe_pointer_to_objref(f_data)::Preconditioner
x = unsafe_wrap(Array, p_x, (n,))
if p_grad !== C_NULL
return data.obj_callback(x, unsafe_wrap(Array, p_grad, (n,)))
end
return data.obj_callback(x, Cdouble[])
end
function nlopt_preconditioner_callback(n, p_x, p_v, p_vpre, f_data)
data = unsafe_pointer_to_objref(f_data)::Preconditioner
x = unsafe_wrap(Array, p_x, (n,))
v = unsafe_wrap(Array, p_v, (n,))
vpre = unsafe_wrap(Array, p_vpre, (n,))
data.preconditioner(vpre, x, v)
return
end
mutable struct Preconditioner
obj_callback::Function
preconditioner::Function
end
Base.unsafe_convert(::Type{Ptr{Cvoid}}, p::Preconditioner) = pointer_from_objref(p)
function precond_max_objective!(opt::NLopt.Opt, obj_fn, precond_fn)::NLopt.Result
preconditioner = Preconditioner(obj_fn, precond_fn)
c_scalar_callback = @cfunction(
nlopt_scalar_callback,
Cdouble,
(Cuint, Ptr{Cdouble}, Ptr{Cdouble}, Ptr{Cvoid}),
)
c_precond_callback = @cfunction(
nlopt_preconditioner_callback,
Cvoid,
(Cuint, Ptr{Cdouble}, Ptr{Cdouble}, Ptr{Cdouble}, Ptr{Cvoid}),
)
return NLopt.nlopt_set_precond_max_objective(
opt,
c_scalar_callback,
c_precond_callback,
preconditioner,
)
end
end
opt = Opt(:LD_CCSAQ, 2)
lower_bounds!(opt, 0.0)
upper_bounds!(opt, 2.0)
xtol_rel!(opt, 1e-4)
function my_objective_fn(x, grad)
if length(grad) > 0
grad[1] = 1.0
grad[2] = 0.5 / sqrt(x[2])
end
return x[1] + sqrt(x[2])
end
function my_preconditioner(vpre, x, v)
vpre[1] = 0.0
vpre[2] = v[2] / (2 * sqrt(x[2]))
return
end
precond_max_objective!(opt, my_objective_fn, my_preconditioner)
min_f, min_x, ret = optimize(opt, [1.0, 1.0])
@test min_f ≈ 2.0 + sqrt(2.0)
@test min_x ≈ [2.0, 2.0]
@test ret == :XTOL_REACHED
Given the docs
Currently, support for preconditioners in NLopt is somewhat experimental, and is only used in the NLOPT_LD_CCSAQ algorithm
I don't know if we need to provide proper support for this.
Although MOI has support for Hessian vector products, so in theory we could hook this up automatically through JuMP.
I found very little documentation about this.
https://nlopt.readthedocs.io/en/stable/NLopt_Reference/#preconditioning-with-approximate-hessians
But that only explains the C interface, and says that only
LD_CCSAQ
supports preconditioners.How is the interface in Julia? Is it possible to pass a preconditioner to LBFGS? Can I pass a sparse Hessian approximation (e.g., a diagonal matrix)?
See also https://discourse.julialang.org/t/how-to-pass-a-preconditioner-to-nlopt/16361