etmc / tmLQCD

tmLQCD is a freely available software suite providing a set of tools to be used in lattice QCD simulations. This is mainly a HMC implementation (including PHMC and RHMC) for Wilson, Wilson Clover and Wilson twisted mass fermions and inverter for different versions of the Dirac operator. The code is fully parallelised and ships with optimisations for various modern architectures, such as commodity PC clusters and the Blue Gene family.
http://www.itkp.uni-bonn.de/~urbach/software.html
GNU General Public License v3.0
32 stars 47 forks source link

re-expose refinenement precision #521

Closed kostrzewa closed 2 years ago

kostrzewa commented 2 years ago

Add back support for writing something like

BeginMonomial NDCLOVERRAT
  Timescale = 1
  kappa = 0.1400645
  CSW = 1.74
  AcceptancePrecision =  1e-21
  ForcePrecision = 1e-16
  StildeMin = 0.0000376
  StildeMax = 4.7
  MaxSolverIterations = 500
  Name = ndcloverrat_0_3
  DegreeOfRational = 10
  Cmin = 0
  Cmax = 3
  ComputeEVFreq = 0
  2Kappamubar = 0.0394421632
  2Kappaepsbar = 0.0426076209
  AddTrLog = yes
  UseExternalInverter = quda
  UseSloppyPrecision = single
  RefinementPrecision = half    # <- refinement precision                                                                                                                                                                                                                                                
  solver = cgmmsnd
EndMonomial

which allows the multi-shift solver to run up to single precision and then refines any unconverged shifts using double-half mixed CG, leading to another 20-25% improvement in time to solution for those monomials.

kostrzewa commented 2 years ago

@simone-romiti can you give this a test run to see if I didn't miss anything?

kostrzewa commented 2 years ago

@simone-romiti can you explain which parts of this you tested before merging? Did you try different combinations of sloppy and reconstruct precisions?

simone-romiti commented 2 years ago

I tried the 9 combinations with A, B = double,single,half in the input file:

... UseSloppyPrecision = A RefinementPrecision = B # <- refinement precision ...
#

kostrzewa commented 2 years ago

Perfect, thank you!