creation of a MYULAnew class

[remilaumont]

closes #406

Notes added Aug 12 2024:

Based on the discussion of design we had with the implicit project group, @remilaumont and I (@amal-ghamdi ) did an iteration of updates and we very much appreciate your review @jakobsj

A few points to keep in mind (based on our implicit project meeting a few days ago and based on email notes that we communicated earlier):

We added a RestorationPrior class as discussed in the design meeting.

• Note that we introduced a parameter restoration_strength which can be passed at initialization. It is also one of the parameters we made required for the function handle for the restorator (what we called: fun, in the email discussion). fun takes, x, restoration_strength and any other parameters the restorator might need.

We added a MoraeuYoshidaPrior class.

• The potential that corresponds to this smoothing is "R_alpha" (not R), thus note that we added implementation for logpdf in MoraeuYoshidaPrior to compute R_alpha (only when R is available because it is based on R)

• At some point in the future, we will also need something similar to MoraeuYoshidaPrior for PnP, as PnP has a more practical appeal in terms of the results it can achieve. One possibility is that we can add a PnPPrior class (not in the scope of this PR) to inherit from MoraeuYoshidaPrior. It can overwrite the logpdf to return Nan but uses the same gradient formula (as both approaches lead to the same gradient formula, although they are coming from two different constructions/derivations)

• Now in principle, MoraeuYoshidaPrior can be tested with MALA/NUTS (suggested by @jeverink at some point), which is really nice, we did not do such tests yet (not necessarily in the scope of this PR but it is nice to try sometime soon)

• Note: a smoothing like the Moraeu Yoshida can in principle be applied to the likelihood (for non-differentiable likelihoods). Is considering the MoraeuYoshidaPrior a prior only, limiting? Should we think of a more generalization? Note that now the MoraeuYoshidaPrior is part of the implicit prior submodule. We might consider relocating it somewhere if we decide to generalize this smoothing to likelihoods.

Compatibility checks for creating a MoraeuYoshidaPrior based on a RestorationPrior

• We needed to check that the smoothing_strength of MoraeuYoshidaPrior is the same as the restoration_strength of the RestorationPrior, and we implemented that check. One thought here is that we can get rid of storing restoration_strength in the RestorationPrior then we will not need this compatibility check. One side effect is that the restore method will need to accept restoration_strength in addition to the other restoration parameters.

• Geometry handling: now we made the MoraeuYoshidaPrior take on the geometry of the passed RestorationPrior. The design of enabling this feature has similarities with what is in the RegularizedGaussian class.

MYULA

• Now, we restrict MYULA to work with RestorationPrior type. We can extend the type of priors it deals with (e.g. any class that has prox) in the future
• MYULA also accepts smoothing_strength parameter at initialization and ensures that it is equal to the restoration_strength of the prior (note that we can get rid of this equality check if we did the suggestion above: not to store the restoration_strength in the RestorationPrior).
• Just a technicality: internally, MYULA now keeps a smoothed target as class attribute (_smoothed_target) for which the gradient is computed when sampling, besides the original target.

[amal-ghamdi]

Hi @nabriis, can you please take an initial look at the initial design of incorporating MYULA. This is the PR by @remilaumont and it is the same PR that we eventually want to merge.

Thank you

[remilaumont]

Thank you @nabriis for your comments. We addressed them and asked again for your review.

[amal-ghamdi]

Many thanks @nabriis for the review!

@jakobsj, would it be possible that you review the PR, much appreciated!