Allow for non standard cosmologies, and improve cosmological time integration

[brantr]

We want to include non-standard cosmological models that have time-dependent dark energy equation of state. This pull request implements w0,wa parameterization of the DE EOS. It also incorporates a Runge-Kutta integrator to improve the integration of the physical timestep with scale factor, pre-computes the offset in physical time for z<\infty ICs, and writes the expansion history to a file. We also allow for non-zero radiation energy density. I have checked the expansion history agrees to within ~0.1% with independent calculations.

[brantr]

The cpp-format test fails on the following code block in global/global.cpp

// NOLINTNEXTLINE(cert-err58-cpp)
// NOLINTNEXTLINE(*)
const std::set<std::string> optionalParams = { "flag_delta","ddelta_dt","n_delta","Lz","Lx","phi","theta","delta","nzr","nxr","H0","Omega_M","Omega_L","Omega_R","Omega_K","w0","wa","Init_redshift","End_redshift", "tile_length", "n_proc_x", "n_proc_y", "n_proc_z" }; // NOLINT

This line already had a NOLINTNEXTLINE(), but I could not get that to work no matter how I formatted the line. I have tried various iterations of NOLINTBEGIN/NOLINTEND, NOLINT, NOLINTNEXTLINE, etc. So if you can recommend how to get around the clang formatting issue, that would be much appreciated.

[mabruzzo]

The cpp-format test fails on the following code block in global/global.cpp

// NOLINTNEXTLINE(cert-err58-cpp)
// NOLINTNEXTLINE(*)
const std::set<std::string> optionalParams = { "flag_delta","ddelta_dt","n_delta","Lz","Lx","phi","theta","delta","nzr","nxr","H0","Omega_M","Omega_L","Omega_R","Omega_K","w0","wa","Init_redshift","End_redshift", "tile_length", "n_proc_x", "n_proc_y", "n_proc_z" }; // NOLINT

This line already had a NOLINTNEXTLINE(), but I could not get that to work no matter how I formatted the line. I have tried various iterations of NOLINTBEGIN/NOLINTEND, NOLINT, NOLINTNEXTLINE, etc. So if you can recommend how to get around the clang formatting issue, that would be much appreciated.

@bcaddy -- do you have thoughts on this?

[brantr]

After a lot of effort and help from Bruno, the version 1f05b8d reproduces well the original cosmological hydro sims using the same flags. The salient issue that may need discussion is the last commit that undoes changes to PPMP_cuda.cu, where ifdef CTU had been changed to ifndef VL. The last commit undoes this change, since it was affecting the cosmological hydro integration, and these portions of the PPMP scheme now only apply to CTU again (not VL or SIMPLE). @evaneschneider do you know why this change was made? Is it OK to keep this off whenever SIMPLE is used, or should I add an ifndef COSMOLOGY and revert the changes back to ifndef VL?

[evaneschneider]

@brantr I do think this needs some discussion. Perhaps part of the problem is that I don't really understand what SIMPLE should do. As I've mentioned a couple times, I think SIMPLE is numerically unstable in more than 1D, since I thought it was just the first half of CTU, and removing the characteristic projections and time evolution of the interface states makes it numerically unstable in 1D as well, so I probably thought I was helping when I noticed that step had been eliminated. (I do remember showing some KH tests in the CAAR slack that demonstrated instabilities that went away when the characteristic projection step was turned back on.) That said, if you all have worked out that simple works without those steps, we can (and perhaps should) get rid of all of that code (and I think we should be consistent here, either we have it in all the reconstructions, or we have it in none). I think resolving this question is best done through a conversation, since it seems like maybe you all did some development work in creating simple that I'm unaware of / do not understand.

[brantr]

Let's discuss offline, but then when we resolve we can post back here for prosperity.

[evaneschneider]

When you get a chance @brantr can you re-run clang-format? I was going to try to investigate the linting error but right now there are a bunch of violations in a bunch of files.