search

LSSTDESC / WLMassMap

DESC utility for weak lensing mass-mapping
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Create scripts for metrics evaluation in mass mapping challenge #8

Open mdschneider opened 7 years ago

mdschneider commented 7 years ago

We need a common set of metrics to compare submissions for the mass mapping challenge (due by 2017-09-15). This issue is write a set of scripts to evaluate the desired metrics. We might also want some associated documentation of the metrics.

From https://github.com/LSSTDESC/WLMassMap/blob/master/doc/challenge.rst the nominal metric ideas are:

M1 RMS error in a selection of random points (suitably smoothed - healpix pixels) M2 Peak/Trough/Void counts (as a function of smoothing scales?) M3 RMS error on points that are masked out M4 Some kind of peak ellipticity metric ?? M5 Power spectrum J1 "Jet" award for best presentation/colour scheme.

jasonmcewen commented 7 years ago

Another useful metric could be the RMS error in annuli away from the centre of the maps. This provides a way to see how the error grows as one approaches the mask boundaries. The metric is discussed in our recent paper (section 4.1, penultimate paragraph) and is implemented in massmappy.