NelderMeadSimplex issue for one-dimensional functions

[MMAvanEnkhuizen]

When I used the MathNet package I found that the function did not work for a simple one dimensional function as 1+x^2. Please see below example where the test OneDimensionalFunction will fail, whereas the two-dimensional version will pass.

Is this intended behaviour?

    [TestFixture]
    public class TestMathNetNelderMead
    {
        private double _tolerance = 1.0e-5;

        [Test]
        public void OneDimensionalFunction()
        {
            NelderMeadSimplex minimiser = new NelderMeadSimplex(1e-10, 500);

            var initialVec = Vector<double>.Build.DenseOfEnumerable(new[] { 1.0 });
            var objFunc = ObjectiveFunction.Value(betas => xSq(betas));

            var min = minimiser.FindMinimum(objFunc, initialVec);
            var xForMinimum = min.MinimizingPoint.ToArray();
            var minimum = xSq(min.MinimizingPoint);

            Assert.AreEqual(1.0, minimum, _tolerance);
            Assert.AreEqual(0.0, xForMinimum[0], _tolerance);
        }

        private double xSq(IEnumerable<double> parameters)
        {
            var beta = parameters.ToArray();
            return 1.0 + System.Math.Pow(beta[0], 2);
        }

        [Test]
        public void TwoDimensionalFunction()
        {
            NelderMeadSimplex minimiser = new NelderMeadSimplex(1e-10, 500);

            var initialVec = Vector<double>.Build.DenseOfEnumerable(new[] { 0.1, 1.0 });
            var objFunc = ObjectiveFunction.Value(betas => xSqPlusYsq(betas));

            var min = minimiser.FindMinimum(objFunc, initialVec);
            var xForMinimum = new[] { min.MinimizingPoint.ToArray()[0], min.MinimizingPoint.ToArray()[1] };
            var minimum = xSqPlusYsq(min.MinimizingPoint);

            Assert.AreEqual(1.0, minimum, _tolerance);
            Assert.AreEqual(0.0, xForMinimum[0], _tolerance);
            Assert.AreEqual(0.0, xForMinimum[1], _tolerance);
        }

        private double xSqPlusYsq(IEnumerable<double> parameters)
        {
            var beta = parameters.ToArray();
            return 1.0 + System.Math.Pow(beta[0], 2) + System.Math.Pow(beta[1], 2);
        }
    }

[eriove]

I implemented/ported this. I simply forgot to test the one dimensional case. Not sure what the intended behavior is. It should either throw an Exception or find the solution. Current behavior is wrong.

On vacation right now. Will check this closer when I have access to my dev environment.

[eriove]

The problem is that the algorithm finds two points at exactly the same distance from the minimum at x=+-0.1. y is then equal to 1.01 for both points and the (currenlty implemented) convergence criteria is fulfilled. I believe this paper describes the convergence criteria that should be used (see 1 in Wikipedia article). Anyone who has access to the paper, or another source describing the convergence criteria for NMS?

[MMAvanEnkhuizen]

I think I found the issue: There seems to be an implementation mistake.

There seems to be some code missing after the reflection. Here there should be also checked that if there happens an shrink or expanding right after this reflection this gives a better optimum.

If this is done, the problem is solves.

[eriove]

Have you tried implementing the fix? If so, could you make a pull request? Unfortunately I have very limited time at the moment to work on this.

[MMAvanEnkhuizen]

Yes I did implement it. Will make a pull request later. Thankyou