Relaxation correctly implemented?

[migarstka]

Wrong implementation of relaxation seems to be the major bug causing the convergence problems!

• Double-check that the relaxation of x and s is correctly implemented. Current implementation for x and s:
• Relaxation of s shouldnt be a problem concerning pos sem-definiteness, since if A, B pos semdef => (A+B) pos semdef

    xt from solving linear system
    xNew = α*xt + (1-α)*xPrev
    st from projection onto pos def cone
    sNew = α*st + (1-α)*sPrev

[migarstka]

The solution follows from Boyd - Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers s.3.4.3. Over-relaxation

In the z- and y-updates the quantity Ax(k+1) can be replaced with alpha*Ax(k+1) - (1-alpha)(Bz(k)-c).

For my ADMM equation the constraint is x=s, hence I do the following:

      xNew = from linear system
      xRelax = α*xNew+(1-α)*sPrev

      #relaxed s and lambda update
       sNew = Projections.sdcone( xRelax + (1/σ)*λPrev,r)
       λNew = λPrev + σ*(xRelax - sNew)