Adding matrix solver and preconditioners

[klausbu]

I am thinking about adding additional linear algebra solvers and preconditioners to Rapid CFD. There are numerous linear algebra libraries that could be leveraged. Each library has it's specific implementation requirements so I have initial questions to get a better understanding of underlying concepts in RapidCFD.

Let's consider the multi GPU case with local data:

• I need to better understand the data structure on the GPUs including upper, diagonal, lower, interface/boundary contributions and how to get hold of the processor interfaces - which files or documents should I look at?

• Is there a known, simple way to convert the ldu data structure on the GPU into the CSR format without doublicating the data and with global indices?

[TonkomoLLC]

Hello,

I have just limited experience here. I ported PBiCGStab to RapidCFD: My work product is located here.

It's also been nearly two years since I did this work, so my memory is a little fuzzy on the details. To the best of my recollection, I started with a CPU (vanilla OpenFOAM) version of PBiCGStab. Then, I found each instance of addition, subtraction, multiplication and division of matrices, using the most similar solver, PBiCG, as a guide on where to make changes.

As a warning: while I did some preliminary testing with icoFoam, I did not test this solver very thoroughly. I basically did this as an exercise to see how one would add a new linear solver.

I did not fully answer your question, but I hope that this practical example will help enable your valued development of new linear solvers and preconditioners. The project will greatly benefit from your developments.

Cheers,

Eric

[klausbu]

Your approach is one way of doing it, what I have in mind is to create an "interface" to matrix solvers and preconditioners provided by external libraries without doubling the memory footprint (LDU format + converted CSR format) on the GPU.

My approach requires to provide the local matrix/data with global indices in CSR format as a starting point.

[TonkomoLLC]

I do not have specific advice for you, but with respect to your earlier query,

which files or documents should I look at?

Probably a starting point is the src/OpenFOAM directory, and in particular, the matrices directory.

I hope this answers your first question. I do not know the answer to your second question, regarding a simple way to convert the ldu data structure on the GPU into CSR format without duplicating the data. Hopefully others are reading this thread and have answers for you.