ILU(0) converge but GMRES preconditioned by ILU and its variant fails to converge

[fw407]

Hello,

I am solving steady RANS equations for compressible viscous flows with finite volume method. I am trying to use GMRES (preconditioned by ILU and its variants) in HYPRE to accelerate my convergence. For some simple cases, it works well. But for some tricky cases, such as flow with separations, GMRES fails to converge. If I use the same Jacobi matrices, but solve it with ILU(0), it actually converges. Below is my setup for GMRES and residual histories for GMRES and ILU(0):

     HYPRE_GMRESSetKDim(gmres_solver, restart);
     HYPRE_GMRESSetMaxIter(gmres_solver, 100); /* max iterations */
     HYPRE_GMRESSetTol(gmres_solver, 1e-1); /* conv. tolerance */
     HYPRE_GMRESSetPrintLevel(gmres_solver, 2); /* print solve info */
     HYPRE_GMRESSetLogging(gmres_solver, 1); /* needed to get run info later */

        ilu_type = 10; //10: gmres-ilu(k), 11: gmres-ilut
        ilu_lfil = 0;
        pc_tol = 0.;
        ilu_max_row_nnz = 1000;
        ilu_droptol = 1.0e-2;
        ilu_schur_max_iter = restart;

      HYPRE_ILUSetType(precond, ilu_type);
      HYPRE_ILUSetLevelOfFill(precond, ilu_lfil);
      HYPRE_ILUSetPrintLevel(precond, 1);
      HYPRE_ILUSetMaxIter(precond, 1);
      HYPRE_ILUSetTol(precond, pc_tol);
      HYPRE_ILUSetMaxNnzPerRow(precond,ilu_max_row_nnz);
      /* set the droptol */
      HYPRE_ILUSetDropThreshold(precond,ilu_droptol);
      /* set max iterations for Schur system solve */
      HYPRE_ILUSetSchurMaxIter( precond, ilu_schur_max_iter );

      HYPRE_GMRESSetPrecond(gmres_solver,
                            (HYPRE_PtrToSolverFcn) HYPRE_ILUSolve,
                            (HYPRE_PtrToSolverFcn) HYPRE_ILUSetup,
                            precond);

The flow field of GMRES is initialized by running ILU(0) for a few hundred iterations and then GMRES is switched on and CFL number is increased gradually to 500.

I have tried to use different variants of ILU (i.e. ILUT, RAS and ILU with different levels of fill) for preconditioning, reduce convergence tolerance, and also increase my krylov space dimension ("restart" in the code snippet) to 1000, they don't not really help.

The Jacobi matrix I give to GMRES is an approximation to the true one, I don't know if this is the problem, but ILU(0) (used as an solver) seems to be happy with it. or the stagnation is caused by something else, Could you please give me any advice for improvement?

Thanks Feng

[oseikuffuor1]

Hi @fw407, When you say ILU(0) converges, do you mean you set ilu_type = 0 in your code snippet or did you just do the solve with the same ILU setup you have in your code, just without GMRES as the accelerator? Also, with just ILU(0), is the Jacobi matrix an approximation of the true one?

-Daniel

[fw407]

Hi @oseikuffuor1 ,

Thanks for your reply. I have a separate function to use ILU (ilu_type=0 and also fill_level = 0) as an iterative solver, without using GMRES as an accelerator. This is normally for me to check if the Jacobi matrix is at least ok or not. The Jacobi matrix is formed in the same way as I use for GMRES, so it is an approximation to the true one.

Thanks, Feng

[oseikuffuor1]

Hi @fw407, So did you try ilu_type = 0 as the preconditioner for GMRES? Did that also fail? Also, how large is the problem and how many processors are you using?

Thanks, -Daniel

[fw407]

Hi @oseikuffuor1

Thanks for your prompt reply. I am using one cpu and the Degree of Freedom (DoF) of the problem is 8036*5. I've tried to set ilu_type to 0 in GMRES, it is doing similar things if ilu_type = 10. I have attached the output of the code when the residual stagnates, hopefully it would be useful.

Thanks, Feng

[oseikuffuor1]

Hi @fw407, If you are using just 1 processor, then the ilu_type options should not matter, and they will all be doing the same thing, which is why you see the similarity between ilu_type 0 and 10. Is the output you shared for GMRES iteration or the ILU(0) solver iteration? The initial convergence rate looks good to me, but it looks like you are doing just 1 iteration, so I am not sure where it stagnates. Is the output you show over multiple Nonlinear solver solves? How many iterations do you do for the ILU(0) stand-alone solver? Perhaps you could try doing the same number of iterations for the preconditioner (just to be sure). The only other thing I see is that you are setting your GMRES tolerance to 0.1. Can you confirm this is the same tolerance used for the ILU(0) solver?

[fw407]

Hi @oseikuffuor1

Sorry for the slow response. I think I might know what the problem is. It is probably due to the defect of the Jacobi I build, ILU and GMRES have different sensitivities to such defect.

The general process I have is that, for each non-linear iteration, it assembles A and b, and I use HYPRE to solve the linear system Ax=b to obtain a solution update x, I update the solution and then go to the next non-linear iteration and so on. The residual history in my first post is the non-linear residual, which is roughly the L2 norm of b. The linear residual is what HYPRE computes, which assess how good Ax=b is solved.

I have tried to set the convergence tolerance of GMRES to machine zero, the linear residual actually converges to machine zero, although the non-linear residual still stagnates. This seems to me that at certain stages the solution update is not right anymore, even though Ax=b is solved without any problem, this is probably due to the defect of A that produces the wrong update using GMRES. With the same Jacobi, using ILU as a standalone solver works, I guess it is because it is an in-complete factorization and somehow relatively insensitive to such defect.

Besides, does HYPRE allows a matrix-free way to do GMRES but uses the approximate Jacobi matrix only for pre-conditioning?

Thanks, Feng

[oseikuffuor1]

Hi Feng, It is actually strange to me that ILU(0) would work as a stand-alone solver for the approximate Jacobian and fail when used as a preconditioner for GMRES, especially if you are using the same options for the preconditioner as the solver. Do you mind adding --with-print-errors to your configure line and rebuilding hypre, then recompile and rerun your code to see if any error messages are printed out. Please share the output of any error messages printed. If no error messages show up, then could you share the solver output printed out for the GMRES run and for the ILU run, just for the one(s) that GMRES fails?

-Daniel

[fw407]

Hi @oseikuffuor1

Thanks for your reply. I have added the flag and re-built HYPRE, it seems to me there is no error messages in the run. I have attached the solver output for GMRES and also ILU. To run GMRES, I first run ILU for 300 iterations and then switch to GMRES. In the output, you may find that the linear residual is fine but the non-linear residual (L2 normal of b) stagnates for GMRES.

Is there a way to use matrix-free GMRES pre-conditioned by an approximate Jacobi matrix in HYPRE?

Thanks, Feng

log_gmres.txt log_ilu.txt

[oseikuffuor1]

Hi @fw407, That's good that no errors were found. Can you confirm that the output (update to the solution) for the 301st iteration was different for both the ILU run and the GMRES run? If you could share the linear systems (Matrix, rhs and solution for both ILU and GMRES, that would be helpful). It looks to me that you set your convergence tolerance for the ILU solver to 1.0e-2 and according to your first post, the GMRES tolerance is set to 1.0e-1? If that is the case, then it is likely to be less accurate. Also, since you are using ILU iteratively, I would suggest using FlexGMRES instead of GMRES.

Unfortunately, the hypre GMRES interface does not support a matrix-free approach.

-Daniel