Mean of deltas is non-zero

[jfarr03]

In the output files from runs of CoLoRe at NERSC, the mean value of all deltas is coming out as a small positive number rather than 0.

The below notebook shows the method of calculating the mean value of delta using an output file generated from a run at NERSC (the params.cfg file that was used for that run is attached). It gives a mean delta over all redshifts of ~0.2.

You can also run the notebook on the example data file stored at example_data/raw_colore/test_N1000.fits of course. (https://github.com/damonge/CoLoRe/files/1502386/test_skewers_4096_gaussian_params.txt)

Notebook: https://github.com/igmhub/LyaCoLoRe/blob/master/example_scripts/issue_mean_delta_colore.ipynb

Parameters file (renamed params.txt): [test_skewers_4096_gaussian_params.txt]

[andreufont]

We talked about it with Javier @fjaviersanchez, and he mentioned it could be related to clipping, but this should only affect the density of tracers, right? The matter density in the skewers should not be affected by any clipping.

[damonge]

I would recommend checking the mean delta over the whole box (which should indeed average to zero)

[andreufont]

I agree. We should check:

• that the mean Gaussian delta is zero, when averaged over the whole box.
• that the mean lognormal density delta is zero, when averaged over the whole box.
• that the mean lognormal density delta as a function of redshift is zero, when averaged over bins in the skewers with the same redshift.

[damonge]

BTW, this is also something that should improve after the revamping. Basically, some of the non-zero mean might be caused by the interpolation from cartesian grid to voxels, which won't be necessary in the future.

Although =0.2 seems huge...

[andreufont]

@damonge I agree.

@jfarr03 - In any case, we will want to have these tools in place because we will need them to compute things like the mean flux as a function of redshift, and so on, so I think it is still worth looking into.

[damonge]

@jfarr03 @andreufont : I've merged the revamp branch into master. 1- Can you check whether this is still a problem in the new version? 2- Can I delete the skewers branch? It's fine if not, just checking.

[andreufont]

Hi @damonge - @jfarr03 and I are meeting tomorrow. We will take a look at it and come back to you. Thanks!

[andreufont]

@damonge - @jfarr03 did a small test with the new code, and it seems to have fixed the issue. Let's wait for a larger test, and then we can close the ticket.

With respect to the skewers branch, he will now compare the correlation function from both branches, and if we get similar results we can close the skewers branch. I believe there was no other feature added there, right?

[damonge]

In principle no. The only issue might be if the problem that caused !=0 also modified the correlation function a little bit, in which case the comparison between branches won't be perfect

[andreufont]

Sure, they will not be exact, since as you pointed out the previous correlation function did not asymptotically approached zero, but rather to a (very small) constant. I just want to make sure that they overall agree, what I'm sure it is the case.

[andreufont]

@damonge - @jfarr03 checked that the two branches give very consistent results. And the mean delta is now very close to zero, so I believe we can close this issue.

[damonge]

That's great, thanks both for checking this!