psi_norm <= 0 error when set a a very strict scf_thr

pxlxingliang commented 6 months ago

Describe the bug

In below system, when I set the scf_thr to 1e-14, the SCF will throw the error:

j = 9 lagrange norm = 1.120e-03
j = 10 lagrange norm = 2.589e-01
j = 11 lagrange norm = 1.108e-03
j = 12 lagrange norm = 4.862e-04
j = 13 lagrange norm = -2.818e-04
j = 14 lagrange norm = -8.445e-05
j = 15 lagrange norm = 1.000e+00
 in DiagoCG, psi norm = -6.710e-02
 If you use GNU compiler, it may due to the zdotc is unavailable.

 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                         NOTICE
 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 psi_norm <= 0.0
 CHECK IN FILE : OUT.ABACUS/warning.log

 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                         NOTICE
 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

And for the similar system, the drho will increase when it reach a small value, and then also throw the psi_norm <= 0 error:

 CG12   -7.836717e+03  9.126054e-10   1.913e-10  1.290e+01
 CG13   -7.836717e+03  -1.611754e-09  1.386e-12  1.561e+01
 CG14   -7.836717e+03  1.392110e-11   2.204e-13  2.496e+01
 CG15   -7.836896e+03  -1.783951e-01  1.427e-05  1.875e+01
 CG16   -7.836717e+03  1.783777e-01   3.575e-12  1.802e+01
 CG17   -7.836983e+03  -2.662777e-01  4.134e-05  2.153e+01
 CG18   -7.836771e+03  2.120962e-01   6.232e-06  1.749e+01
 m = 38
j = 0 lagrange norm = -2.776e-01
j = 1 lagrange norm = -1.477e-01
j = 2 lagrange norm = -7.781e-02
j = 3 lagrange norm = 5.110e-03
...
j = 38 lagrange norm = 1.000e+00
 in DiagoCG, psi norm = -2.373e-01
 If you use GNU compiler, it may due to the zdotc is unavailable.

 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                         NOTICE
 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 psi_norm <= 0.0
 CHECK IN FILE : OUT.ABACUS/warning.log

 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
                         NOTICE
 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
TIME STATISTICS

psinorm.zip

Expected behavior

No response

To Reproduce

No response

Environment

No response

Additional Context

No response

Task list for Issue attackers (only for developers)

[ ] Verify the issue is not a duplicate.
[ ] Describe the bug.
[ ] Steps to reproduce.
[ ] Expected behavior.
[ ] Error message.
[ ] Environment details.
[ ] Additional context.
[ ] Assign a priority level (low, medium, high, urgent).
[ ] Assign the issue to a team member.
[ ] Label the issue with relevant tags.
[ ] Identify possible related issues.
[ ] Create a unit test or automated test to reproduce the bug (if applicable).
[ ] Fix the bug.
[ ] Test the fix.
[ ] Update documentation (if necessary).
[ ] Close the issue and inform the reporter (if applicable).

WHUweiqingzhou commented 6 months ago

maybe @kirk0830 have a look if the same problem exists in QE?

kirk0830 commented 6 months ago

I test conv_thr keyword with value 1e-06, 1e-10, 1e-12, to 1e-20, QE can converge wavefunction well. Finally I directly set to 1e-100 and find scf_accurancy osillates in level about 1e-25, then exit normally due to cannot get converged to the given threshold within maximal scf steps.

&CONTROL
calculation = 'scf'
outdir = './output/'
prefix = 'test'
pseudo_dir = './'
restart_mode = 'from_scratch'
wf_collect = .TRUE.
/

&SYSTEM
degauss = 0.002
ecutwfc = 30
ibrav = 0
nat = 1
ntyp = 1
occupations = 'smearing'
/

&ELECTRONS
electron_maxstep = 100
mixing_beta = 0.5
scf_must_converge = .TRUE.
conv_thr = 1e-100
/

ATOMIC_SPECIES
Si 28.0855 Si_ONCV_PBE-1.0.upf

K_POINTS {automatic}
6 6 6 0 0 0

CELL_PARAMETERS {angstrom}
2.6406597000 0.0000000000 0.0000000000
1.3203298500 2.3008051588 0.0000000000
1.3203298500 0.0000000000 2.3008051588

ATOMIC_POSITIONS {angstrom}
Si 1.980494775 1.6977995643363967e-16 1.1504025794166375

kirk0830 commented 6 months ago

Even for diagonalization = 'cg', QE did not crash:

     total cpu time spent up to now is        3.2 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          1.2E-17 Ry

     iteration # 94     ecut=    30.00 Ry     beta= 0.50
     CG style diagonalization
     ethr =  1.00E-13,  avg # of iterations =  3.0

     total cpu time spent up to now is        3.2 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          3.1E-18 Ry

     iteration # 95     ecut=    30.00 Ry     beta= 0.50
     CG style diagonalization
     ethr =  1.00E-13,  avg # of iterations =  3.0

     total cpu time spent up to now is        3.3 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          4.0E-18 Ry

     iteration # 96     ecut=    30.00 Ry     beta= 0.50
     CG style diagonalization
     ethr =  1.00E-13,  avg # of iterations =  3.0

     total cpu time spent up to now is        3.3 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          4.7E-19 Ry

     iteration # 97     ecut=    30.00 Ry     beta= 0.50
     CG style diagonalization
     ethr =  1.00E-13,  avg # of iterations =  3.0

     total cpu time spent up to now is        3.3 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          8.8E-19 Ry

     iteration # 98     ecut=    30.00 Ry     beta= 0.50
     CG style diagonalization
     ethr =  1.00E-13,  avg # of iterations =  3.0

     total cpu time spent up to now is        3.4 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          2.7E-19 Ry

     iteration # 99     ecut=    30.00 Ry     beta= 0.50
     CG style diagonalization
     ethr =  1.00E-13,  avg # of iterations =  3.0

     total cpu time spent up to now is        3.4 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          8.5E-19 Ry

     iteration #100     ecut=    30.00 Ry     beta= 0.50
     CG style diagonalization
     ethr =  1.00E-13,  avg # of iterations =  3.0

     total cpu time spent up to now is        3.4 secs

     total energy              =      -7.83826961 Ry
     estimated scf accuracy    <          2.3E-18 Ry

     End of self-consistent calculation

     convergence NOT achieved after 100 iterations: stopping

WHUweiqingzhou commented 6 months ago

@haozhihan, could you have a look?

haozhihan commented 5 months ago

You can use dav_subspace method and it won't trigger this problem. Everything is very good in dav_subspace method for pw basis.

haozhihan commented 5 months ago

ABACUS v3.6.0

               Atomic-orbital Based Ab-initio Computation at UStc                    

                     Website: http://abacus.ustc.edu.cn/                             
               Documentation: https://abacus.deepmodeling.com/                       
                  Repository: [GitHub - abacusmodeling/abacus-develop: An electronic structure package based on either plane wave b](https://github.com/abacusmodeling/abacus-develop)       
                              [GitHub - deepmodeling/abacus-develop: An electronic structure package based on either plane wave bas](https://github.com/deepmodeling/abacus-develop)         
                      Commit: 21d40ce93 (Tue Apr 9 22:54:12 2024 +0800)

 Thu Apr 11 18:23:06 2024
 MAKE THE DIR         : OUT.ABACUS/
 RUNNING WITH DEVICE  : CPU / Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
 Warning: the number of valence electrons in pseudopotential > 3 for Ga: [Ar] 3d10 4s2 4p1
 Pseudopotentials with additional electrons can yield (more) accurate outcomes, but may be less efficient.
 If you're confident that your chosen pseudopotential is appropriate, you can safely ignore this warning.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

 UNIFORM GRID DIM        : 54 * 54 * 54
 UNIFORM GRID DIM(BIG)   : 54 * 54 * 54
 DONE(0.0372433  SEC) : SETUP UNITCELL
 DONE(0.0451674  SEC) : INIT K-POINTS
 ---------------------------------------------------------
 Self-consistent calculations for electrons
 ---------------------------------------------------------
 SPIN    KPOINTS         PROCESSORS  
 1       172             16          
 ---------------------------------------------------------
 Use plane wave basis
 ---------------------------------------------------------
 ELEMENT NATOM       XC          
 Ga      4           
 As      4           
 ---------------------------------------------------------
 Initial plane wave basis and FFT box
 ---------------------------------------------------------
 DONE(0.0565177  SEC) : INIT PLANEWAVE
 MEMORY FOR PSI (MB)  : 78.1107
 DONE(0.073572   SEC) : LOCAL POTENTIAL
 DONE(0.156863   SEC) : NON-LOCAL POTENTIAL
 DONE(0.156907   SEC) : INIT BASIS
 -------------------------------------------
 SELF-CONSISTENT : 
 -------------------------------------------
 START CHARGE      : atomic
 DONE(0.217457   SEC) : INIT SCF
 ITER   ETOT(eV)       EDIFF(eV)      DRHO       TIME(s)    
 DS1    -7.833699e+03  0.000000e+00   3.488e-01  1.385e+02  
 DS2    -7.835485e+03  -1.786125e+00  1.318e-02  1.397e+01  
 DS3    -7.836593e+03  -1.107802e+00  2.570e-02  5.051e+01  
 DS4    -7.836673e+03  -8.004766e-02  2.251e-03  1.312e+01  
 DS5    -7.836703e+03  -3.004028e-02  9.265e-04  2.045e+01  
 DS6    -7.836712e+03  -8.534544e-03  3.021e-05  1.409e+01  
 DS7    -7.836713e+03  -1.285950e-03  1.055e-04  1.123e+01  
 DS8    -7.836714e+03  -1.253721e-03  2.474e-06  1.115e+01  
 DS9    -7.836715e+03  -6.918219e-04  5.752e-07  1.114e+01  
 DS10   -7.836715e+03  -4.850237e-04  5.134e-07  1.114e+01  
 DS11   -7.836716e+03  -3.543776e-04  8.440e-07  1.115e+01  
 DS12   -7.836716e+03  -3.465173e-04  5.565e-06  1.120e+01  
 DS13   -7.836716e+03  -2.692743e-04  3.263e-06  1.114e+01  
 DS14   -7.836716e+03  -1.844974e-04  2.344e-07  1.114e+01  
 DS15   -7.836717e+03  -1.387245e-04  1.711e-07  1.113e+01  
 DS16   -7.836717e+03  -1.127373e-04  1.313e-08  1.114e+01  
 DS17   -7.836717e+03  -9.371561e-05  6.248e-09  1.115e+01  
 DS18   -7.836717e+03  -7.432137e-05  1.667e-08  1.120e+01  
 DS19   -7.836717e+03  -5.972532e-05  5.913e-08  1.129e+01  
 DS20   -7.836717e+03  -4.873905e-05  7.258e-08  1.120e+01  
 DS21   -7.836717e+03  -3.898438e-05  5.982e-08  1.117e+01  
 DS22   -7.836717e+03  -3.071734e-05  8.501e-08  1.116e+01  
 DS23   -7.836717e+03  -2.490417e-05  8.611e-08  1.120e+01  
 DS24   -7.836717e+03  -1.936493e-05  1.971e-07  1.117e+01  
 DS25   -7.836717e+03  -1.582482e-05  1.879e-07  1.119e+01  
 DS26   -7.836717e+03  -1.240883e-05  2.202e-07  1.110e+01  
 DS27   -7.836717e+03  -9.991222e-06  1.569e-07  1.112e+01  
 DS28   -7.836717e+03  -7.922242e-06  1.301e-07  1.115e+01  
 DS29   -7.836717e+03  -6.449330e-06  7.336e-08  1.115e+01  
 DS30   -7.836717e+03  -5.519708e-06  1.549e-08  1.112e+01  
 DS31   -7.836717e+03  -4.602826e-06  2.745e-09  1.112e+01  
 DS32   -7.836717e+03  -3.811373e-06  1.658e-10  1.112e+01  
 DS33   -7.836717e+03  -3.175659e-06  2.289e-11  1.113e+01  
 DS34   -7.836717e+03  -2.684257e-06  9.118e-11  1.119e+01  
 DS35   -7.836717e+03  -2.253698e-06  4.569e-10  1.112e+01  
 DS36   -7.836717e+03  -1.923401e-06  7.468e-10  1.110e+01  
 DS37   -7.836717e+03  -1.651530e-06  1.042e-09  1.112e+01  
 DS38   -7.836717e+03  -1.408906e-06  1.552e-09  1.111e+01  
 DS39   -7.836717e+03  -1.199959e-06  1.990e-09  1.119e+01  
 DS40   -7.836717e+03  -1.011369e-06  2.461e-09  1.111e+01  
 DS41   -7.836717e+03  -8.437686e-07  1.951e-09  1.112e+01  
 DS42   -7.836717e+03  -7.085298e-07  1.555e-09  1.111e+01  
 DS43   -7.836717e+03  -6.043551e-07  1.991e-09  1.111e+01  
 DS44   -7.836717e+03  -5.209507e-07  2.339e-09  1.110e+01  
 DS45   -7.836717e+03  -4.401743e-07  2.544e-09  1.117e+01  
 DS46   -7.836717e+03  -3.824528e-07  2.624e-09  1.116e+01  
 DS47   -7.836717e+03  -3.315372e-07  2.900e-09  1.113e+01  
 DS48   -7.836717e+03  -2.832804e-07  2.629e-09  1.112e+01  
 DS49   -7.836717e+03  -2.372573e-07  1.774e-09  1.112e+01  
 DS50   -7.836717e+03  -1.972790e-07  1.188e-09  1.119e+01  
 DS51   -7.836717e+03  -1.740153e-07  3.049e-10  1.112e+01  
 DS52   -7.836717e+03  -1.519581e-07  1.187e-11  1.112e+01  
 DS53   -7.836717e+03  -1.356193e-07  4.974e-13  1.113e+01  
 DS54   -7.836717e+03  -1.201252e-07  1.012e-12  1.112e+01  
 DS55   -7.836717e+03  -1.049543e-07  3.869e-12  1.115e+01  
 DS56   -7.836717e+03  -9.537345e-08  5.946e-12  1.114e+01  
 DS57   -7.836717e+03  -8.444848e-08  6.903e-12  1.112e+01  
 DS58   -7.836717e+03  -7.665885e-08  8.167e-12  1.109e+01  
 DS59   -7.836717e+03  -6.855368e-08  1.149e-11  1.108e+01  
 DS60   -7.836717e+03  -6.043149e-08  1.575e-11  1.109e+01  
 DS61   -7.836717e+03  -5.385300e-08  2.009e-11  1.115e+01  
 DS62   -7.836717e+03  -4.600459e-08  2.118e-11  1.109e+01  
 DS63   -7.836717e+03  -4.140135e-08  2.211e-11  1.109e+01  
 DS64   -7.836717e+03  -3.510282e-08  1.137e-11  1.109e+01  
 DS65   -7.836717e+03  -2.950500e-08  6.004e-12  1.111e+01  
 DS66   -7.836717e+03  -2.774784e-08  8.608e-12  1.116e+01  
 DS67   -7.836717e+03  -2.403864e-08  1.132e-11  1.108e+01  
 DS68   -7.836717e+03  -1.987778e-08  6.757e-12  1.111e+01  
 DS69   -7.836717e+03  -1.915853e-08  6.304e-12  1.110e+01  
 DS70   -7.836717e+03  -1.679658e-08  3.428e-13  1.110e+01  
 DS71   -7.836717e+03  -1.486773e-08  6.443e-13  1.109e+01  
 DS72   -7.836717e+03  -1.395977e-08  8.378e-15  1.115e+01  
----------------------------------------------------------------
TOTAL-STRESS (KBAR)                                           
----------------------------------------------------------------
      -10.1403768952         0.0000657977         0.0000117003
        0.0000657977       -10.1410813338         0.0000523706
        0.0000117003         0.0000523706       -10.1404519885
----------------------------------------------------------------
 TOTAL-PRESSURE: -10.140637 KBAR

TIME STATISTICS
-------------------------------------------------------------------------------------
     CLASS_NAME                 NAME             TIME(Sec)  CALLS   AVG(Sec) PER(%)
-------------------------------------------------------------------------------------
                     total                       995.09          15  66.34   100.00
Driver               reading                       0.01           1   0.01     0.00
Input                Init                          0.01           1   0.01     0.00
Input_Conv           Convert                       0.00           1   0.00     0.00
Driver               driver_line                 995.08           1 995.08   100.00
UnitCell             check_tau                     0.00           1   0.00     0.00
PW_Basis_Sup         setuptransform                0.00           1   0.00     0.00
PW_Basis_Sup         distributeg                   0.00           1   0.00     0.00
mymath               heapsort                      0.00           3   0.00     0.00
PW_Basis_K           setuptransform                0.01           1   0.01     0.00
PW_Basis_K           distributeg                   0.00           1   0.00     0.00
PW_Basis             setup_struc_factor            0.00           1   0.00     0.00
ppcell_vnl           init                          0.00           1   0.00     0.00
ppcell_vl            init_vloc                     0.01           1   0.01     0.00
ppcell_vnl           init_vnl                      0.08           1   0.08     0.01
WF_atomic            init_at_1                     0.00           1   0.00     0.00
wavefunc             wfcinit                       0.00           1   0.00     0.00
Ions                 opt_ions                    994.90           1 994.90    99.98
ESolver_KS_PW        run                         985.80           1 985.80    99.07
H_Ewald_pw           compute_ewald                 0.04           1   0.04     0.00
Charge               set_rho_core                  0.00           1   0.00     0.00
Charge               atomic_rho                    0.01           1   0.01     0.00
PW_Basis_Sup         recip2real                    0.21         441   0.00     0.02
PW_Basis_Sup         gathers_scatterp              0.09         441   0.00     0.01
Potential            init_pot                      0.01           1   0.01     0.00
Potential            update_from_charge            0.63          73   0.01     0.06
Potential            cal_fixed_v                   0.00           1   0.00     0.00
PotLocal             cal_fixed_v                   0.00           1   0.00     0.00
Potential            cal_v_eff                     0.63          73   0.01     0.06
H_Hartree_pw         v_hartree                     0.07          73   0.00     0.01
PW_Basis_Sup         real2recip                    0.26         590   0.00     0.03
PW_Basis_Sup         gatherp_scatters              0.11         590   0.00     0.01
PotXC                cal_v_eff                     0.55          73   0.01     0.06
XC_Functional        v_xc                          0.55          73   0.01     0.06
Potential            interpolate_vrs               0.00          73   0.00     0.00
Charge_Mixing        init_mixing                   0.00           1   0.00     0.00
ESolver_KS_PW        hamilt2density              984.86          73  13.49    98.97
HSolverPW            solve                       984.86          73  13.49    98.97
Nonlocal             getvnl                       19.92       12556   0.00     2.00
pp_cell_vnl          getvnl                       20.44       12900   0.00     2.05
Structure_Factor     get_sk                        1.79       18404   0.00     0.18
DiagoIterAssist      diagH_subspace                8.13         344   0.02     0.82
Operator             hPsi                        648.98       34844   0.02    65.22
Operator             EkineticPW                    2.47       34844   0.00     0.25
Operator             VeffPW                      556.08       34844   0.02    55.88
PW_Basis_K           recip2real                  397.64     1929304   0.00    39.96
PW_Basis_K           gathers_scatterp            106.93     1929304   0.00    10.75
PW_Basis_K           real2recip                  255.31     1351728   0.00    25.66
PW_Basis_K           gatherp_scatters             53.76     1351728   0.00     5.40
Operator             NonlocalPW                   90.26       34844   0.00     9.07
Nonlocal             add_nonlocal_pp              33.39       34844   0.00     3.36
DiagoIterAssist      diagH_LAPACK                  0.24         344   0.00     0.02
Diago_DavSubspace    diag_once                   820.41       12556   0.07    82.45
Diago_DavSubspace    first                       304.12       12556   0.02    30.56
Diago_DavSubspace    cal_elem                     34.98       34500   0.00     3.51
Diago_DavSubspace    diag_zhegvx                  89.98       34500   0.00     9.04
Diago_DavSubspace    cal_grad                    388.39       21944   0.02    39.03
Diago_DavSubspace    check_update                  0.02       21944   0.00     0.00
Diago_DavSubspace    last                         16.89       17594   0.00     1.70
Diago_DavSubspace    refresh                       4.55        5038   0.00     0.46
ElecStatePW          psiToRho                    114.67          73   1.57    11.52
Charge_Mixing        get_drho                      0.08          73   0.00     0.01
Charge_Mixing        inner_product_recip_rho       0.00          73   0.00     0.00
Charge               mix_rho                       0.14          71   0.00     0.01
Charge               Broyden_mixing                0.05          71   0.00     0.00
Charge_Mixing        inner_product_recip_hartree   0.04        1064   0.00     0.00
Forces               cal_force_loc                 0.00           1   0.00     0.00
Forces               cal_force_ew                  0.00           1   0.00     0.00
Forces               cal_force_nl                  1.37           1   1.37     0.14
Forces               cal_force_cc                  0.00           1   0.00     0.00
Forces               cal_force_scc                 0.01           1   0.01     0.00
Stress_PW            cal_stress                    7.72           1   7.72     0.78
Stress_Func          stress_kin                    0.14           1   0.14     0.01
Stress_Func          stress_har                    0.00           1   0.00     0.00
Stress_Func          stress_ewa                    0.00           1   0.00     0.00
Stress_Func          stress_gga                    0.00           1   0.00     0.00
Stress_Func          stress_loc                    0.02           1   0.02     0.00
Stress_Func          stress_cc                     0.00           1   0.00     0.00
Stress_Func          stress_nl                     7.56           1   7.56     0.76
ModuleIO             write_istate_info             0.02           1   0.02     0.00
-------------------------------------------------------------------------------------

 START  Time  : Thu Apr 11 18:23:06 2024
 FINISH Time  : Thu Apr 11 18:39:41 2024
 TOTAL  Time  : 995
 SEE INFORMATION IN : OUT.ABACUS/

WHUweiqingzhou commented 5 months ago

I made some tests. Indeed dav_subspace (PR #3903) would not trigger this issue.

deepmodeling / abacus-develop