search

LSSTDESC / DEHSC_LSS

Scripts and workflow relevant to HSC data
1 stars 3 forks source link

Solving #30 #31

Closed fjaviersanchez closed 5 years ago

fjaviersanchez commented 6 years ago

This PR addresses #30.

Now check_sys is able to ingest multi-bin flatmaps. It outputs a series of plots like the one below:

screen shot 2018-10-10 at 4 57 40 pm

These plots live in the output directory set by the user.

It can generate histograms with equally spaced bins, log-spaced bins or population weighted bins (i.e, binning using the percentiles of the systematic map to analyze -- this is the default)

It has been tested at NERSC as follows: python check_sys.py --output-prefix $HOME/test_HSC_sys/ --nsys-bins 10 --map-path /global/cscratch1/sd/damonge/HSC/HSC_processed_TAN/WIDE_AEGIS/WIDE_AEGIS_Ngal_bins_eab_best_single.fits --input-prefix /global/cscratch1/sd/damonge/HSC/HSC_processed/WIDE_AEGIS/WIDE_AEGIS

@damonge Do we have a directory with the maps that contain more than one redshift bin?

Any feedback/requests are welcome!

damonge commented 6 years ago

@fjaviersanchez awesome! Forget about the HSC_processed_TAN folder. The 4-bin maps can be found at e.g. /global/cscratch1/sd/damonge/HSC/HSC_processed/WIDE_AEGIS/WIDE_AEGIS_Ngal_bins_eab_best_4pzbins.fits

fjaviersanchez commented 6 years ago

This is now working with the file containing 4 redshift bins (bin 0) 😄

screen shot 2018-10-22 at 12 20 00 pm
fjaviersanchez commented 6 years ago

For the record I am leaving here the command that I used to get this (and other) plots.

python check_sys.py --input-prefix /global/cscratch1/sd/damonge/HSC/HSC_processed/WIDE_AEGIS/WIDE_AEGIS --map-path /global/cscratch1/sd/damonge/HSC/HSC_processed/WIDE_AEGIS/WIDE_AEGIS_Ngal_bins_eab_best_pzb4bins_pdfstack.fits --depth-cut 24.5 --output-prefix $HOME/HSC_sys_tests

damonge commented 5 years ago

@fjaviersanchez this is great. Would it be possible to plot also Delta(n) as a function of Delta(syst), where n is the number of objects per pixel, syst is a given systematic and Delta(x) is the difference of x with respect to the mean?

fjaviersanchez commented 5 years ago

@damonge I just did that. Please, feel free to take a look at the code and check for further improvements. Thanks!

screen shot 2018-11-08 at 11 25 49 am

damonge commented 5 years ago

Hey @fjaviersanchez

this is really great. I've made a small commit with only a few small changes:

  1. Fixed the issue with percentiles, and removed the logarithmic option (I thought that wasn't super useful, and also had some problems with division by zero)
  2. The code now saves the data themselves in npz files as well as the plots, in case we need to look at the actual numbers
  3. I've integrated the systematics checks with the rest of the pipeline.

If you agree with the changes feel free to merge!