search

JuliaNLSolvers / Optim.jl

Optimization functions for Julia
Other
1.11k stars 213 forks source link

Solving functions in which it's possible to get Inf and NaN values #1000

Closed Boxylmer closed 2 months ago

Boxylmer commented 1 year ago

I have a function which is difficult to bound (predicting bounds is difficult and often not easily defined in the parameter space), and some out-of-bounds values will return Inf, missing, or NaN (I can replace these with a single type if need be). I've found that when a good starting point is known, NelderMead and BFGS optimize well. Is there a way to have one of these handle Inf values in some way, shape or form? Ideally I'd like to have the optimizer "jump back" or otherwise restrict the search from the areas in which these values were returned. If not, I guess I'll be using SAMIN!

sallyc1997 commented 1 year ago

Hi! I'm also in a similar situation. I'm using Optim.optimize function to perform maximum likelihood estimation where I could get log(0) in some cases. I tried both NelderMead() and BFGS() and I see that they both can continue to optimize after seeing -Inf. How exactly does Optim.optimize function handle those -Inf cases?

pkofod commented 1 year ago

Well, if you just make sure that your objective does not throw an error, Inf or NaN values will cause line search methods to reject the step length, and take a shorter step. Nelder Mead vil do something similar. There you have a handful of different types of mutations we do to the simplex, and if one causes an Inf to be return, that value will never be "good enough" for us to accept the step, so we will try another mutation, and in the end we will shrink the simplex.

Everything could work well is long as you don't start with a non-finite value of the objective or in the gradient elements, or as long as you don't end up in a situation where the objective itself is finite but the gradient has some sort of non-finitevalues.