Bug in regularization

[timueh]

Thanks to @Alexdaijlu we found a bug in

https://github.com/alexe15/ALADIN.m/blob/1639fabf9fa66c5ec54642e63cac789a97d675f9/src/core/regularizeH.m#L24

Solution: replace transpose(V) by V'.

[xinliang-dai]

Thanks to @Alexdaijlu we found a bug in

https://github.com/alexe15/ALADIN.m/blob/1639fabf9fa66c5ec54642e63cac789a97d675f9/src/core/regularizeH.m#L24

Solution: replace transpose(V) by V.

I guess you miss a ', replace by V'

[timueh]

lol, yeah!

[xinliang-dai]

I went through the Eigenvalue Modification in section 3,4 from Numerical Optimization. It mentions some strategies to modify Hessian, including add smalles eigenvalue and flip the eigenvale, which is used in regularizeH().

They takes add smalles eigenvalue as an example and set the opts.regParam to 1e-8, while the default value in our Toolbox, which is assigned to reg in regularizeH.m, is 1e-4.

Here I have a try with opts.regParam = 1e-8 in Morenet Project, and the results are following:

   ========================================================      
   ==               This is ALADIN-M v0.1                ==      
   ========================================================      
   Algorithm:        ALADIN
   QP solver:        sparseBs
   Local solver:     ipopt
   Inner algorithm:  none

   Given termination tolerance:1e-10
   Consensus violation:Inf    0.06299011   0.002338445   1.10972e-05  1.829598e-09  4.230838e-12
   Maximum number of iterations is reached without satisfying given termination criterion.

   -----------------   ALADIN-M timing   ------------------
                       t[s]        %tot         %iter
   Tot time:......:    5.6                                 
   Prob setup:....:    1.0         18.4                    
   Iter time:.....:    4.5         81.2                    
   ------
   NLP time:......:    2.1                      46.9       
   QP time:.......:    0.6                      13.7       
   Reg time:......:    0.3                      7.3        
   Plot time:.....:    1.3                      29.0       

   ========================================================

ALADIN Algorithm converged in 5 iterations
comparison_aladin =

  3×2 table

    Regions    Inf-Norm of residual
    _______    ____________________

       1            3.9013e-11     
       2            3.3607e-12     
       3            1.0669e-13   

I think regularizeH(), based on flip the eigenvale, with opts.regParam = 1e-8, works well in this specific problem. And we don't need to skip the regularization step in Morenet Project.

@timueh @alexe15

PS: this is 3X118 casefile again.

[timueh]

Interesting observation, but think of Occam's razor: if we see satisfactory convergence without regularization, why bother adding it?

[xinliang-dai]

Interesting observation, but think of Occam's razor: if we see satisfactory convergence without regularization, why bother adding it?

In this specific case, it works well without regularization. But I'm not for sure whether it would also converge, without regularization, for others, or larger-scale cases.