Refactor tmLQCD after qphix merge

[kostrzewa]

@martin-ueding @plabus @pittlerf @urbach @marcus-petschlies @Finkenrath @sbacchio

Carsten, you mentioned that you would like to discuss some form of refactoring for tmLQCD. Would there be interest amongst all of you to collaborate in this?

[urbach]

Indeed, I would be in favour of it. In particular the memory management. We should come up with a priority list.

[kostrzewa]

Okay, this needs to be done sooner rather than later, otherwise supporting QPhiX and QUDA in the HMC will consist of hacks upon hacks once again... It is essential that we define consistent interfaces for all solvers.

[martin-ueding]

Would you like to use an object oriented (abstract base class) kind of interface for this?

[kostrzewa]

Would you like to use an object oriented (abstract base class) kind of interface for this?

I don't think it will be possible to go quite that far given that we're dealing with C code and would need to write C-style calling code anyway.

The main problem is as follows: Up until 2014, the policy was that solvers for monomials were called essentially directly in the monomial code. This is fine if you don't use any external libraries and simply pass the correct function pointer to the solver (doing some pre- and post-processing, if required).

In 2014 Florian Burger introduced an interface which leads to some abstraction. This allowed him to call his GPU solvers in the HMC. It worked reasonably well because his solvers were implemented in tmLQCD.

However, when support for DDalphaAMG was added, it already broke the pattern. Once called, the solver has to process the calling parameters once again to set itself up correctly and this involves relying on comparisons of function pointers and parsing of global variables. Nothing wrong with that in essence, but it becomes very messy once QUDA and QPhiX are added as further possibilities with all their respective parameters and inconsistent interfaces.

There's also the issue of partial support for non-even-odd pre-conditioned monomials (which Florian needed for finite temperature calculations, to be able to run odd-numbered time extents).

[kostrzewa]

Okay, I think I have a rough idea how this should proceed now. Basically, because we are using solver libraries which have relatively clean and consistent interfaces, we need to have the same kind of clean interface on the tmLQCD side, otherwise everything is a pain.

1) remove Florian's GPU stuff 1) combine solver parameters into a struct 1) single interface for clover and non-clover inversions 1) single interface for hmc and regular solves 1) move even/odd source preparation to its own function (for uses where this is required) 1) move even/odd reconstruction to its own function (for uses where this is required)

This will induce very substantial changes across the code-base and will also require a rethink of the DDalphaAMG interface, but I don't see an alternative at this point. I hope that doing this will enable us to also more or less easily use QUDA solvers in the HMC in the end.

[kostrzewa]

In a second stage:

1) add pool memory manager 1) remove all global spinor fields