search

ralna / RALFit

A non-linear least squares solver
Other
25 stars 6 forks source link

evaltensor_f #71

Open talassio opened 4 years ago

talassio commented 4 years ago

Under the context of Tensor-Newton Subproblem.

Multiple redundant calls to eval_HF when eval_HP is present

Problem

In the recursive call to evaltensor_hf there is a redundant call to eval_hf at x=params%x that does not change withing a recursive call to nlls_solve.

Fix

Add flag to avoid calling uneccesarily eval_hf within evaltensor_hf...

Showcase

Lanczos.f90 with or without eval_hp call back

     16 5.8309E-08  7.62945E-05  2.23415E-01  1.64E+06  9.18E-01 STRC     358 1.9E-01 TR Ok    1.00
---------------------------------------------------------------------------------------------------
      0 5.8309E-08  7.62945E-05  2.23415E-01  1.00E+02 -1.00E+00 --I-
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
      1 5.7811E-08  4.47801E-07  1.31694E-03  2.00E+02  1.00E+00 SNI-         3.0E-04 TR Ok    1.00
      2 5.7643E-08  2.20741E-06  6.50120E-03  4.00E+02  9.98E-01 SYI-         4.5E-03 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
      3 5.7642E-08  7.82994E-08  2.30607E-04  8.00E+02  1.00E+00 SNI-         1.9E-05 TR Ok    1.00
      4 5.7369E-08  5.09497E-06  1.50414E-02  1.60E+03  9.94E-01 SYI-         7.2E-03 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
      5 5.7364E-08  7.30301E-08  2.15610E-04  3.20E+03  1.00E+00 SNI-         6.6E-05 TR Ok    1.00
      6 5.6852E-08  1.59537E-05  4.73121E-02  6.40E+03  9.70E-01 SYI-         1.4E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
      7 5.6823E-08  7.21911E-08  2.14144E-04  1.28E+04  1.00E+00 SNI-         2.5E-04 TR Ok    1.00
      8 5.6041E-08  5.53252E-05  1.65256E-01  1.28E+04  8.02E-01 SYI-         2.6E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
      9 5.5820E-08  6.48920E-08  1.94214E-04  2.56E+04  1.00E+00 SNI-         4.0E-04 TR Ok    1.00
     10 5.5153E-08  8.99576E-05  2.70855E-01  2.56E+04  5.69E-01 SYI-         3.3E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     11 5.4592E-08  5.74673E-08  1.73917E-04  5.12E+04  1.00E+00 SNI-         7.2E-04 TR Ok    1.00
     12 5.4359E-08  1.26257E-04  3.82916E-01  5.12E+04  1.91E-01 SYI-         4.0E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     13 5.3273E-08  5.57465E-08  1.70784E-04  1.02E+05  1.00E+00 SNI-         1.2E-03 TR Ok    1.00
     14                                       5.12E+04 -1.47E-01 UYI-
     14 5.2840E-08  8.05000E-05  2.47627E-01  5.12E+04  5.42E-01 SYI-         3.3E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
---------------------------------------------------------------------------------------------------
  Iter |   error  |   optim    |  rel optim |  Delta  |   rho   |S2IF| inn it|  step | loop|LS| tau
---------------------------------------------------------------------------------------------------
     15 5.2388E-08  3.63556E-08  1.12316E-04  1.02E+05  1.00E+00 SNI-         9.4E-04 TR Ok    1.00
     16 5.2150E-08  7.90295E-05  2.44708E-01  1.02E+05  4.18E-01 SYI-         3.4E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     17 5.1713E-08  3.68390E-08  1.14549E-04  2.05E+05  1.00E+00 SNI-         1.3E-03 TR Ok    1.00
     18 5.1539E-08  4.75545E-05  1.48118E-01  2.05E+05  6.69E-01 SYI-         2.7E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     19 5.1375E-08  8.87601E-08  2.76902E-04  4.10E+05  1.00E+00 SNI-         1.4E-03 TR Ok    1.00
     20 5.1296E-08  1.44657E-05  4.51628E-02  8.19E+05  1.16E+00 SYI-         1.6E-02 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     21 5.1273E-08  3.62754E-07  1.13280E-03  1.64E+06  9.94E-01 SNI-         2.4E-03 TR Ok    1.00
     22 5.1260E-08  1.99739E-06  6.23819E-03  3.28E+06  1.40E+00 SYI-         6.0E-03 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     23 5.1257E-08  3.54128E-07  1.10603E-03  6.55E+06  9.49E-01 SNI-         2.4E-03 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     24 5.1257E-08  8.25167E-08  2.57722E-04  1.31E+07  1.42E+00 SYI-         1.3E-03 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     25 5.1257E-08  1.85295E-08  5.78727E-05  2.62E+07  1.43E+00 SYI-         5.6E-04 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     26 5.1257E-08  3.52720E-09  1.10164E-05  5.24E+07  1.44E+00 SYI-         2.5E-04 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     27 5.1257E-08  6.72223E-10  2.09953E-06  1.00E+08  1.44E+00 SYI-         1.1E-04 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     28 5.1257E-08  1.28500E-10  4.01340E-07  1.00E+08  1.00E+00 SYI-         4.7E-05 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     29 5.1257E-08  2.48728E-11  7.76846E-08  1.00E+08  1.00E+00 SYI-         2.1E-05 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     30 5.1257E-08  5.09597E-12  1.59161E-08  1.00E+08  1.00E+00 SYI-         9.0E-06 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     31 5.1257E-08  1.26741E-12  3.95847E-09  1.00E+08  1.00E+00 SYI-         3.9E-06 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     32 5.1257E-08  4.29920E-13  1.34276E-09  1.00E+08  1.00E+00 SYI-         1.7E-06 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     33 5.1257E-08  1.75667E-13  5.48656E-10  1.00E+08  1.00E+00 SYI-         7.5E-07 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     34 5.1257E-08  7.57051E-14  2.36447E-10  1.00E+08  1.00E+00 SYI-         3.3E-07 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     35 5.1257E-08  3.29861E-14  1.03025E-10  1.00E+08  1.00E+00 SYI-         1.4E-07 TR Ok    1.00
 EVALUATION Hf @ x=  0.25892606574455329        1.5139338761396632        1.8028587114687327        4.0864347691744571       0.45168112936543126        6.0083834535144192     
     36 5.1257E-08  1.44034E-14  4.49858E-11  1.00E+08  1.00E+00 SYI-         6.2E-08 TR Ok    1.00
---------------------------------------------------------------------------------------------------
 EVALUATION Hf @ x=  0.24610105492175122        1.4851846891680101        1.7053653236113633        4.0085490010973119       0.56199265759382211        5.8198897602272197     
     17 5.0828E-08  1.75838E-04  5.51500E-01  1.64E+06  7.22E-01 STRC     394 2.5E-01 TR Ok    1.00
...
...
...
----------------------------------------------------------
  Status: converged, an optimal solution was found
 ----------------------------------------------------------
 Norm of error                      8.05672E-09
 Norm of gradient                   1.42339E-13
 Norm of scaled gradient            1.12132E-09
 Iteration count                             30
     Trust region step                       30 (100.0%)
     LS step                                  0 (  0.0%)
     PG step                                  0 (  0.0%)
 Function evaluations                        31
     Trust region step                       31 (100.0%)
     LS step                                  0 (  0.0%)
     PG step                                  0 (  0.0%)
 Gradient evaluations                        31
     Trust region step                       31 (100.0%)
     LS step                                  0 (  0.0%)
     PG step                                  0 (  0.0%)
 Hessian evaluations (eval_hf)              628   <----- this number should be 31
 Hessian evaluations (eval_hp)              958

Multiple redundant calls to eval_HP

At every inner iteration 0, the nlls_iterate is started with s=x0=0.0, which will trigger an expensive call to eval_HP or or m matrix-vector products with zero-vector y that ultimately will return a zero matrix...

Either check if all(y==0) at the entry of calculate_sHs. A good option is to not call tensoreval_f when we already have the answer s=0 then f(s) = r_k