ADMM Hyper-parameters

[utsav-akhaury]

Current ADMM hyper-parameters lead to an optimal reconstruction with NMSE ~7%, the set of choices being taken from this implementation of the ADMM Deep Plug&Play paper.

Ideally, after reaching the optimal reconstruction, the algorithm should converge and give the same output for subsequent iterations. However, in both the NumPy and TensorFlow implementations, the solution diverges after a certain number of iterations (as shown in this notebook).

Although early stopping can resolve this issue, there might be a better set of hyper-parameters that lead to convergence. Hence, the aim is to tune the ADMM update hyper-parameters in such a way that the output converges to the optimal solution.

[utsav-akhaury]

• The Gradient step size update (α) is updated using the Spectral Norm (SV) of the operator, which is calculated using the power iteration method.

• The update equation - *α = 1 / ((SV + ρσ²) (1 + 10^-5))**

• The direct TensorFlow implementation (below) gives a similar output to the power iteration method (the order of difference being ~10^-4)

  H = tf.signal.fft2d(psf)
  normH = tf.math.abs(tf.reverse(tf.reverse(H, axis=[0]), axis=[1]) * H + rho*sigma**2)
  return tf.math.reduce_max(normH)

• The value of ρ (the augmented lagrangian parameter) does not change after a few iterations (because it reaches the upper cap defined in the ADMM Deep Plug&Play code). Hence, SV & α do not change subsequently.

[utsav-akhaury]

Convergence achieved by updating the Gradient step size update (α) as discussed in the comment mentioned above Both NumPy and TensorFlow give similar results (order of difference ~10-4)