search

andreufont / LyaCosmoParams

Notes on cosmological parameters and Lyman alpha simulations
2 stars 2 forks source link

Restructure postprocessing functions #98

Open Chris-Pedersen opened 3 years ago

Chris-Pedersen commented 3 years ago

These are currently hardcoded with DiRAC submission scripts. I have scripts to run postprocessings on Hypatia that are not currently integrated with the repo, but at some point we should decide how to restructure these, i.e. just hardcode the functions with Hypatia scripts now, or come up with some more generalisable framework.

andreufont commented 3 years ago

Hi @Chris-Pedersen - One of the things that has been in my to-do list for a while now is to move some of the simulations to the local cluster in Barcelona. I might finally have time to do this during the next couple of weeks, and I could use the oportunity to update these scripts (I want to make sure I know how to run them myself).

andreufont commented 3 years ago

@Chris-Pedersen - I'm already at the point where I'd like to run post-processing for a suite of sims, and I would like to see what scripts you have in Hypatia to do this. Each machine might need somewhat different setups, but that's ok.

Chris-Pedersen commented 3 years ago

Hi @andreufont I've pushed the changes I made to get the postprocessings to work on Hypatia to a PR #104 which will point you to the parts of the code that need to be modified to run on PIC. This will all need to be tidied up a bit in the future as lots of things are hardcoded. For example the spectra.json files have the original DiRAC directories for each snapshot hardcoded into them as well, which is redundant information now. Parts of the postprocessing code attempted to read this which I have edited out for now.

andreufont commented 3 years ago

Thanks. I'll take a look at that, it will probably be enough to get started.

Chris-Pedersen commented 3 years ago

A summary of the changes to be made to the postprocessing code:

  1. Calculate p1d along all 3 axes and take the average.
  2. Have one place to store a template submission script, that will then be used to generate and submit scripts for each stage of the postprocessings.
  3. Set up scripts that will run the postprocessings for individual sims as well as the LH suite. This will make it easier to add the postprocessings for new test sims as we go.

I have made a start on 2 and 3, and happy to take over the PR #107 to address 1.

andreufont commented 3 years ago

1) Hi @Chris-Pedersen - If possible, I'd try to treat the 3 axes similarly as to how we treat the plus/minus boxes. We have 6 "views" of the same universe. Note that each of these 6 views will have slightly different mean flux as well, and we want to take that into account when combining P1Ds (see what we do for plus / minus). An important difference is that variables that are computed from the box (and not from the skewers) are the same in the three axes, like T0, gamma and k_F.

2) and 3) make sense.