Implementation roadmap

[jiahao]

A comprehensive listing of methods in the references. Please remove any methods are impractical/obsolete and add methods worth implementing to the list.

Linear systems

• Stationary methods
  • [x] Jacobi (LT)
  • [x] Gauss-Seidel (LT)
  • [x] Successive overrelaxation, SOR (LT)
  • [x] Symmetric SOR (LT)
• Nonstationary methods
  • [x] GMRES (LT, Saa)
  • [x] MINRES (LT)
  • [x] Conjugate Gradients, CG (LT)
  • [x] LSQR
  • [x] LSMR
  • [x] BiCGStab(l) (Sle)
  • [x] Chebyshev iteration (LT)
  • [ ] Quasi-Minimal Residual, QMR (LT)
  • [ ] DQGMRES

(Generalized) eigenproblems

• Without subspace
  • [x] (Shift-and-inverted) power iteration (ET)
  • [ ] Rayleigh Quotient Iteration [1]
• Krylov subspace methods
  • [ ] Implicitly restarted Lanczos (ET)
  • [ ] Implicitly restarted Arnoldi (ET)
• Other subspace methods
  • [ ] Jacobi-Davidson method (ET)
  • [x] LOPCG
  • [ ] Accelerated Rayleigh Quotient Iteration [1]
  • [ ] (Shift-and-inverted) subspace iteration (ET)

Singular value decomposition

• [x] Golub-Kahan-Lanczos (ET)

References

• LT: Templates for the Solution of Linear Systems
• ET: Templates for the Solution of Algebraic Eigenvalue Problems
• Saa: GMRES: A generalized minimal residual algorithm for solving nonsymmetric linear systems (pdf)
• Sle: BiCGstab(l) and other hybrid Bi-CG methods (pdf)
• TB: Numerical linear algebra
• HV: Iterative Krylov methods for large linear systems

[1] Low-priority methods: they are quite expensive per iteration.

[cortner]

@andreasnoack Thank you - I agree it would be good to stick with this.

[raminammour]

Hello all!

First thank you, I am a frequent user/abuser of this package! (I hope this is a good thread for this post)

1- I was wondering if you have any plans to support the LinearMaps.jl package as a general interface, instead of MatrixFcn which tries to accomplish the same, with less options. I have been using LinearMaps instead of MatrixFcn with IterativeSolvers, and for the most part, it works.

2- When the preconditioners are passed as a function (as in gmres for example), it is better to assume that the inverse is passed directly so Pinv*x replaces Pl\x in the code, or have the two options, maybe. For most of the things I do, the linear operators are large scale and beyond building explicitly.

3- I also cannot wait till this package is integrated with different optimization packages to replace all direct solvers. It will open many doors for large scale optimization.

Again, thanks for the developers, Cheers!

[syntapy]

Hey, I'd like to work on this for GSOC 2017 if yalls would be fine with that. I think I'd have fun with it. I'm not sure yet whether I'd be able to do GSOC full time since I have work obligations, but if not, I'd still love to do this part time for free

[haampie]

Hi all,

I would like to work on this project as well for GSoC 2017. What are the current priorities? Implementing more iterative solvers? Integration with LinearMaps.jl?

Some ideas I have: implementing (eigen)solvers listed above such as Jacobi-Davidson, BiCGStab(l), IRAM (but there are already some old & open PRs on this); an maybe add some new flavours to existing methods like [thick restarted / augmented subspace / deflated] GMRES. All these latter methods are based on the idea that the Krylov subspace obtained at the end of a cycle contains very useful spectral data which can be exploited in the next cycle; potentially restoring superlinear convergence behaviour observed in full GMRES.

[barche]

From my point of view (as a potential user) I would be interested most in preconditioners (ILU, AMG).

[erantreister]

Hi Bart, I have a package with some multigrid preconditioners in Multigrid.jl. In particular smoothed aggregation AMG. It's a part of the JuliaInv collection of packages. Cheers, Eran Treister.

[barche]

Aha, good to know, thanks!

[GregVernon]

How about the recently developed Alternating Anderson Richardson (AAR) method?

Alternating Anderson-Richardson method: An efficient alternative to preconditioned Krylov methods for large, sparse linear systems https://arxiv.org/pdf/1606.08740.pdf

[mohamed82008]

It would be nice to have an updated roadmap in light of the new developments in Julia's iterative solvers ecosystem.

[kiranshila]

Just submitted a PR for QMR