Non convergence of `norm`

[benjohnbarnes]

I found this odd:

julia> Optim.optimize(norm, [1.0, 1.0]) Results of Optimization Algorithm

• Algorithm: Nelder-Mead
• Starting Point: [1.0,1.0]
• Minimum: [0.16666666666666666,0.16666666666666666]
• Value of Function at Minimum: 0.235702
• Iterations: 4
• Convergence: true
  • |x - x'| < NaN: false
  • |f(x) - f(x')| / |f(x)| < 1.0e-08: true
  • |g(x)| < NaN: false
  • Exceeded Maximum Number of Iterations: false
• Objective Function Calls: 9
• Gradient Call: 0

However, initial vectors slightly different from [1.0, 1.0] do converge with a minimum of 0.0, or very close to 0.0.

[mlubin]

Nelder-Mead has very weak convergence guarantees. Use a gradient-based algorithm if at all possible.

[johnmyleswhite]

@benjohnbarnes: I'm really sorry that I keep closing your issues, but I think you should pause for a second and think about whether you really want to post things that get seen by so many people who have work that you're distracting them from. I think you're misusing community resources by opening issues without thinking very deeply about what you're saying.

For this example, you should make sure that you've read papers on Nelder-Mean before opening an issue. You can't have an informed opinion about what's odd until you know what's normal. What you're seeing is normal behavior from Nelder-Mead.

I'm very sorry to be so blunt with you. But I receive literally thousands of e-mails from GitHub issues every day and it's become impossible to triage them effectively. I need you to help do some of the triage yourself.

[benjohnbarnes]

Hi @johnmyleswhite, no problem about bluntness. I understand entirely. I hope I'll get a change to look the technique up and get some understanding of it.

[benjohnbarnes]

Thank you @johnmyleswhite. I had a look and I see that Nelder Mead is considered to have poor convergence properties.

If Optim is purely intended for optimisation researchers who understand the convergence criteria and other pros and cons of the various algorithms, then what I'm about to say isn't important, but perhaps the Readme should actively deter potential ignorants like myself from getting too stuck in.

From the Readme, I assumed Optim is (also) aimed at end users that do not have a developed understanding of optimisation, but simply want a black box that will mostly work. As an analogy – spreadsheets can be used by anyone, not just accountants.

So, If Optim is intended for relatively naive end users like me, can I suggest that the API choose another default algorithm in place of Nelder Mead? Or, provide a textual warning about the default algorithm's limited convergence and perhaps a brief general link for anyone interested to educate themself?

I think either, or both of these; or alternatively a warning in the Readme to scare away the naive, would help you receive fewer emails to triage.

Thank you.

[mlubin]

I'm not opposed to a warning when nelder-mead is chosen as the default, maybe after https://github.com/JuliaOpt/Optim.jl/pull/149 lands.