If Wolfe conditions not met, conjugateGradient runs through all iterations silently...

[RobertLRead]

I think if the Wolfe conditions are not met and "a" in the loop below is zero, the code simply runs through maxIterations without reporting it or making in progress. If the user is unaware of this, it effectively creates an infinite loop where no progress is made.

I'm not sure how to solve this, but I would recommend simply "failing fast" by through an error if "a" is returned as 0 from wolfeLineSearch. Through an error back and let the user deal with it.

for (var i = 0; i < maxIterations; ++i) { a = wolfeLineSearch(f, pk, current, next, a);

        // todo: history in wrong spot?
        if (params.history) {
            params.history.push({x: current.x.slice(),
                                 fx: current.fx,
                                 fxprime: current.fxprime.slice(),
                                 alpha: a});
        }

// console.log("XXX",i,current.x,current.fx,a);

    // NOTE: I think there is a bug here.  If a == 0, then we run throguh
    // all iterations without making any change at all; we are not using i.
    // If the Wolfe conditions fail, we should abort immediately or do something else.
        if (!a) {
            // faiiled to find point that satifies wolfe conditions.
            // reset direction for next iteration
            scale(pk, current.fxprime, -1);

        } else {
            // update direction using Polak–Ribiere CG method
            weightedSum(yk, 1, next.fxprime, -1, current.fxprime);

            var delta_k = dot(current.fxprime, current.fxprime),
                beta_k = Math.max(0, dot(yk, next.fxprime) / delta_k);

            weightedSum(pk, beta_k, pk, -1, next.fxprime);

            temp = current;
            current = next;
            next = temp;
        }

        if (norm2(current.fxprime) <= 1e-5) {
            break;
        }
    }

    if (params.history) {
        params.history.push({x: current.x.slice(),
                             fx: current.fx,
                             fxprime: current.fxprime.slice(),
                             alpha: a});
    }

    return current;
}

[artix41]

I had also noticed the bug with conjugate gradients and Wolfe linear search, but couldn't find its origin. Thanks for your solution, I'm gonna try that later!

[Jylomaki]

I'm also ran into this issue. I saw that the strong Wolfe condition is here in order to ensure a maximal step for an efficient gradient descent.

I'm not sure as to what the issue is related to, but I made my code works by doing this kind of thing:

gradient = gradient / fx // my gradient is now in the [0,1] range
for i in gradient.length:
   gradient[i] = gradient[i] * eigenValue[i];

This ensure that my gradient contains value that won't get me out of my space search. (I had a big loss function: sum of square distances ~10K on eval and a small space search: unit normed deformable shape & rotation in [-1,1] and [-Pi,Pi])