Integrate CHARM into ltu-cmass

[maho3]

This task is to implement CHARM into the ltu-cmass pipeline.

CHARM is a normalizing flow model to connect the dark matter density field with voxelized halo counts and masses. In recent tests, it has shown the ability to reproduce the unbiased halo 2-pt functions with <10% error.

The primary goal of this task would be to create a step in the pipeline which implements a trained CHARM model, following the convention of rho_to_halo.py.

It will also probably require a comparable training script, similar to fit_halo_bias.py

[maho3]

@shivampcosmo

[maho3]

Updates here: https://docs.google.com/presentation/d/1hBMhlSDsBqP2lJ1GtU1KxeXi9vd6VE11bveA5EssQnI/edit#slide=id.p

[maho3]

CHARM reproducing the the halo positions in a BORGPM run

Ongoing issues:

• [x] Currently using z=50 for IC input to CHARM, BORG CLASS broken beyond that
• [ ] Currently randomly assigning subvoxel halo positions, we can perhaps interpolate with sample_3d
• [ ] Memory problems may exist in the higher volume sims. To be tested

[maho3]

CHARM is now integrated into the ltu-cmass in the charm_matt branch. It uses borg_pm snapshots and initial conditions (at z=50), and corrects the linear density with a colossus growth factor to be equivalent to z=99 (the trained input of CHARM).

It works for some cases... Here's an example of the quijote lhid=3 simulation:

However, there is still an underestimation of clusters:

Also, the clustering results fail more notably for other cosmologies. For example, lhid=2:

It's not clear whether this is a result of misinputs, or cosmology dependence, or a broader problem with CHARM.

One thing of note, is this behavior is also apparent when using Shivam's premade fastPM inputs. For example, again for lhid=3:

[maho3]

It turns out the previous comparisons were comparing CHARM outputs to the Quijote FoF catalogs, whereas CHARM was trained to reproduce Quijote Rockstar. When comparing them to the right Quijote things, the HMF looks consistent but the Power is still off...

[maho3]

After fixing meshing:

[maho3]

Modified Shivam's code to patch CHARM to 2 Gpc/h

[maho3]

Todo:

• [ ] Test CHARM reconstruction at different redshift
• [ ] Test P(k) comparison at 2 Gpc/h volumes with theory and Abacus
• [ ] Test and improve sub-voxel positions and velocities