Objective passed to NelderMead callback does not match computed objective

[taranraj123]

It appears that the objective value passed to the callback in NelderMead() does not match objective when evaluated in the callback. Here is a MWE of the issue,

using Optimization
using OptimizationOptimJL

rosenbrock(x,p) =  (p[1] - x[1])^2 + p[2] * (x[2] - x[1]^2)^2
x0 = zeros(2)
p  = [1.0,100.0]

prob = OptimizationProblem(rosenbrock,x0,p)

function callback(x, obj)
           println(obj)
           println(rosenbrock(x, p))
           println("---")
           false
end

sol = solve(prob, NelderMead(); callback=callback)

The test environment looks like below,

(jl_OiGC2u) pkg> st
Status `/private/var/folders/vn/_zsgk6k508507wjmjtfmptnc00020x/T/jl_OiGC2u/Project.toml`
  [7f7a1694] Optimization v3.11.0
  [36348300] OptimizationOptimJL v0.1.5

[Vaibhavdixit02]

That's because the obj is not necessarily the current function evaluation but the current minimum and is stored in the state https://github.com/JuliaNLSolvers/Optim.jl/blob/master/src/multivariate/solvers/zeroth_order/nelder_mead.jl#L204. You can recreate this with only Optim as well

julia> using Optim

julia> f(x) = (1.0 - x[1])^2 + 100.0 * (x[2] - x[1]^2)^2
f (generic function with 1 method)

julia> x0 = [0.0, 0.0]
2-element Vector{Float64}:
 0.0
 0.0

julia> function callback(state)
                  println(state.value)
                  println(f(state.metadata["centroid"]))
                  println("---")
                  false
       end
callback (generic function with 2 methods)

julia> Optim.optimize(f, x0, NelderMead(), Optim.Options(;callback = callback, extended_trace = true))
0.9506640624999999
0.9751586914062501
---
0.9506640624999999
0.9751586914062501
---
0.9506640624999999
0.9612558078765869
---
0.9262175083160399
0.9244886499643326
---
0.8292372137308122
0.8738775933161378
---
0.8292372137308122
0.8473658752441406
---
0.8138906955718994
0.8006465918198228
---
0.7569606018066407
0.7592333227396011
---
0.7382898432016374
0.7221220219507813
---
0.6989375501871109
0.6884825229644775
---
0.6800415039062501
0.6576585388556124
---
0.6475000095367432
0.6291676622629166
---
0.6377041727304458
0.6027013653889299
---
0.5977620035409928
0.5781249999999999
---
0.5977620035409928
0.5554777974262833
---
0.54761962890625
0.5727641815319657
---
0.5449374371790326
0.5364276940563286
---
0.5091263252892531
0.5256378991180098
---
0.5091263252892531
0.5121865607966993
---
0.45644345282505117
0.47934235962071453
---
0.45644345282505117
0.4512577769487689
---
0.3653678089009192
0.4041019491593857
---
0.3653678089009192
0.36546268684674027
---
0.2993955551182538
0.3296388616713656
---
0.2993955551182538
0.31960404461114417
---
0.25977773590695985
0.27288023040721754
---
0.22658525620344372
0.232906778103922
---
0.14448917088453422
0.17927999309268483
---
0.14448917088453422
0.17465768295391332
---
0.14448917088453422
0.1436263047453315
---
0.09866232875869388
0.11740070179560216
---
0.09866232875869388
0.10870631166082632
---
0.06483338481155441
0.07899520361850604
---
0.06483338481155441
0.07676747778685743
---
0.054847295607669294
0.054300787700446954
---
0.048796232743291285
0.0376251725861657
---
0.012198511818044756
0.02529572385197833
---
0.012198511818044756
0.019052864859973335
---
0.012198511818044756
0.01336522747852669
---
0.005976420650439886
0.008035907488525763
---
0.005976420650439886
0.006318252588808773
---
0.0026241908943930767
0.0021631367837034567
---
0.0026241908943930767
0.0038680961697054052
---
0.0026241908943930767
0.0022452816305392623
---
0.00035825898674718686
0.0009915336124318192
---
0.00035825898674718686
0.00023888572912242144
---
4.826749182911818e-5
6.001275086620938e-5
---
4.826749182911818e-5
1.8434997507689553e-5
---
4.826749182911818e-5
7.517971217861088e-6
---
3.58990668454171e-5
1.6377798130657373e-6
---
1.3316611971068665e-5
1.7813727887091015e-5
---
1.5652129074070739e-6
1.8613586102007604e-6
---
1.5652129074070739e-6
3.704747034521236e-6
---
1.5652129074070739e-6
7.827713718572145e-7
---
1.5652129074070739e-6
2.840121408613159e-7
---
4.966342403850991e-7
2.817696114696205e-7
---
2.272706709692903e-7
1.2308485767164133e-7
---
2.120248852111823e-7
3.1501503862751765e-8
---
6.942358374981032e-8
4.340673504142109e-8
---
1.876333755675703e-8
1.4145738345114643e-8
---
1.876333755675703e-8
3.7357441674966805e-9
---
 * Status: success

 * Candidate solution
    Final objective value:     3.525527e-09

 * Found with
    Algorithm:     Nelder-Mead

 * Convergence measures
    √(Σ(yᵢ-ȳ)²)/n ≤ 1.0e-08

 * Work counters
    Seconds run:   0  (vs limit Inf)
    Iterations:    60
    f(x) calls:    117

[taranraj123]

I see. Thanks for clarifying this!