A shortcoming of the current implementation is that the user must provide the step sizes when the algorithm begins, and the step sizes are scaled uniformly across all dimensions as the algorithm proceeds. If the user chooses the scales poorly or if the relative curvature of dimensions change significantly, then the convergence rate will likely suffer. I would like to support adaptive step sizes where doubling or halvings can be applied to only some dimensions. There are several procedures to do so in the literature, so this will require doing some research.
It's worth noting that uniform scaling of the step sizes is a valuable feature (sufficient condition) for convergence guarantees (unlike, say, the Nelder-Mead simplex algorithm).
A shortcoming of the current implementation is that the user must provide the step sizes when the algorithm begins, and the step sizes are scaled uniformly across all dimensions as the algorithm proceeds. If the user chooses the scales poorly or if the relative curvature of dimensions change significantly, then the convergence rate will likely suffer. I would like to support adaptive step sizes where doubling or halvings can be applied to only some dimensions. There are several procedures to do so in the literature, so this will require doing some research.
It's worth noting that uniform scaling of the step sizes is a valuable feature (sufficient condition) for convergence guarantees (unlike, say, the Nelder-Mead simplex algorithm).