Write N-body ICs to a file

[marcelo-alvarez]

Adds the ICs class with a method to write particle positions, masses and velocities, suitable for use in N-body sims. See comment below for more details.

[marcelo-alvarez]

@1cosmologist this is ready for you to review (not requesting for you to validate accuracy of the output files).

Running

srun -n 16 mockgen mockgen-5 --N 2048 --Niter 1 --laststep writeics --icw --zInit 50

on 4 GPU nodes (16 GPUs total) of Perlmutter in the directory

/global/cfs/cdirs/mp107d/exgal/users/malvarez/tests/mockgen

generates the files

/global/cfs/cdirs/mp107d/exgal/users/malvarez/tests/mockgen/output/mockgen-5_13579_Lbox-7700_N-2048_proc[0-15]

@jmsexton03 can you have a look and see if these files look ok? If so then we can look at running Nyx on these outside of this issue. They should correspond to the ICs for LCDM cosmology at z=50 in a 7700 Mpc box at 2048^3 resolution, for the following cosmological parameters passed to CAMB:

            'T_cmb': 2.72548,
            'h': 0.68,
#=================Omegas========================
            'Omega_m': 0.31,      # Matter density fraction
            'Omega_b': 0.0493890, # baryon density fraction
            'Omega_k': 0.,
#=============Spectral params===================
            'A_s': 2.10058e-9,   # scalar amplitude
            'n_s': 0.96605,      # scalar spectral index
            'alpha_s': 0.,       # scalar spectral running
            'k_pivot': 0.05,     # pivot scale for scalar perturbations
            'r': 0.,             # tensor to scalar ratio
            'n_t': 0.,           # tensor spectral index
            'alpha_t': 0.,       # tensor spectral running

#=============reionization========================
            'tau_reion': 0.0543,    # optical depth at reionization
            'YHe': 0.2454,          # Helium fraction
#=============Light relics===========================
            'N_ur': 2.0328,     # Ultra-relativistic species (ultra-light neutrinos)
            'N_ncdm': 1,        # Light massive relics including massive neutrinos
            'm_ncdm': 0.06,     # mass of non-cold light relics
#===========DE EOS===================================
            'w0': -1.,         
            'wa': 0.

[1cosmologist]

@marcelo-alvarez Can you add the details of the Nyx output format. It would be useful to have it documented or linked as a part of this PR.

[marcelo-alvarez]

@1cosmologist sure here is the documentation for Nyx format: https://amrex-astro.github.io/Nyx/docs_html/ICs.html#start-from-a-binary-file

[marcelo-alvarez]

See comment inline to review about creating a new issue focused on generic interfaces for this functionality.

[1cosmologist]

Create new issues in xgfield and cosmology to add velocity and Hubble_parameter methods. But these codes should not be a part of mockgen.

[marcelo-alvarez]

Thanks @1cosmologist for the review. This code needs to be re-written in order to address the issues you raised, so I'm closing this PR without merging. I will not delete the branch in case some of the code in it can be reused in a new implementation.