lattice / quda

QUDA is a library for performing calculations in lattice QCD on GPUs.
https://lattice.github.io/quda
Other
289 stars 97 forks source link

Direct aggregation staggered MG fails #1340

Closed weinbe2 closed 1 year ago

weinbe2 commented 1 year ago

In the current develop branch, staggered MG fails in half during the block orthogonalize path, where the norms of the block-ortho'd fields coming out as zero. This is low priority because there are no issues with the Kahler-Dirac preconditioned workflow, but nonetheless needs to be fixed.

Reproducer command:

ARGSTR="--prec double --prec-sloppy single --prec-null half --prec-precondition half --recon 13 --recon-sloppy 9 --recon-precondition 9"
ARGSTR="$ARGSTR --dim 8 8 8 8 --gridsize 1 1 1 1 "
ARGSTR="$ARGSTR --dslash-type asqtad --compute-fat-long true --tol 1e-10 --mass 0.1"
ARGSTR="$ARGSTR --verbosity verbose --solve-type direct --solution-type mat --inv-type gcr"
ARGSTR="$ARGSTR --inv-multigrid true --mg-levels 2 --mg-coarse-solve-type 0 direct"
ARGSTR="$ARGSTR --mg-staggered-coarsen-type aggregate --mg-use-mma true"
#ARGSTR="$ARGSTR --mg-block-size 0 1 1 1 1 --mg-nvec 0 3"
ARGSTR="$ARGSTR --mg-block-size 0 4 4 4 4 --mg-nvec 0 64"
ARGSTR="$ARGSTR --nsrc 1 --niter 25"
ARGSTR="$ARGSTR --mg-setup-tol 0 1e-6 --mg-setup-maxiter 0 10 --mg-setup-inv 0 cgnr "
ARGSTR="$ARGSTR --mg-smoother 0 ca-gcr --mg-smoother-solve-type 0 direct --mg-nu-pre 0 0 --mg-nu-post 0 8"
ARGSTR="$ARGSTR --mg-smoother 1 ca-gcr --mg-smoother-solve-type 1 direct --mg-nu-pre 1 0 --mg-nu-post 1 8"
ARGSTR="$ARGSTR --mg-coarse-solver 1 gcr --mg-coarse-solve-type 1 direct --mg-coarse-solver-tol 1 0.25 --mg-coarse-solver-maxiter 1 8"
ARGSTR="$ARGSTR --mg-coarse-solver 2 gcr --mg-coarse-solve-type 2 direct-pc --mg-coarse-solver-tol 2 0.25 --mg-coarse-solver-maxiter 2 8"
ARGSTR="$ARGSTR --mg-verbosity 0 verbose --mg-verbosity 1 verbose --mg-verbosity 2 verbose "

export QUDA_RESOURCE_PATH=`pwd`/tunecache

mpirun -np 1 ./staggered_invert_test $ARGSTR

Error:

weinbe2 commented 1 year ago

This is fixed in #1333