Split Grid modifications to work with Chroma

[bjoo]

Outline

This PR makes split-grid work with additions to Chroma including with QDP-JIT. The main list of modifications to the existing split grid setup are:

• Mechanism for saving/restoring gauge fields and clover fields before and after the split
  • NB: this takes gauge and clover fields out of the interface to the invertMultiSrc....() functions. The existing loaded gauge fields are used (in internal native format)
• ColorSpinorFields are 'downloaded' and converted to native format prior to splitting. This adds an extra 'download' since the underlying call to invertQuda will also then 're-download' (now from device-to-device) and also this uses some extra memory. This could be avoided by refactoring invertQuda to take existing ColorSpinorField-s and provide an overloaded version as a wrapper which takes the host pointers to satisfy the C-intarface. The wrapper could do the downloads and call the newly refactored invertQuda. That is a potentially more involved refactor -- because of things like resident solutions etc -- so save for a later (next?) PR.
• A PR to exploit this in Chroma is ready, but I don't want to raise it until this PR is merged, since it needs this PR to work.

[bjoo]

Hi @maddyscientist -- Hopefully I managed to resolve all your comments, I left open the conversation about CPS changing. I also noted that cscs builds failed their Ctest, so that is something I will dig into -- I have been testing solely through Chroma for these changes so far.