search

jennykim1016 / OM10

Tools for working with the Oguri & Marshall (2010) mock catalog of strong gravitational lenses
MIT License
0 stars 0 forks source link

Fix Reweighting algorithm #6

Closed jennykim1016 closed 7 years ago

jennykim1016 commented 7 years ago

The gaussian_reweight method does not calculate the weight properly; I should fix it so that the mean of the redshift in OM10 lenses matches that of LRGs.

jennykim1016 commented 7 years ago

Normalizing the histogram has no problem as of now, but the weighting still has problem. Especially,

fig1 = corner.corner(features, labels=labels, color='blue', weight=True, smooth=1.0, hist_kwargs=dict(normed=True)) fig2 = corner.corner(features, labels=labels, color='blue', smooth=1.0, hist_kwargs=dict(normed=True))

the resulting cornerplots from those two lines are identical. Rejection sampling did not make any difference. It might be great to investigate the corner() function call and the method itself.

drphilmarshall commented 7 years ago

I think weight has to be set to a numpy array of the same length as the features array, so that each sample gets a weight. I'm surprised the corner code runs when you pass it just a single boolean!

Test: if you plot your weights against their feature (eg sample['ZLENS']) values using plt.plot(), you should see them lie underneath an envelope whose shape you derived...

jennykim1016 commented 7 years ago

Oh, if I run the code above, as you pointed out, it might not run. That is not a real code that I have been using -- the real code that I've been using is either fig1 = om10.plot_figure(features, labels=labels, color='blue', weight=True, smooth=1.0, hist_kwargs=dict(normed=True)) or fig1 = corner.corner(features, labels=labels, color='blue', weights=db.sample['weight'], smooth=1.0, hist_kwargs=dict(normed=True)). Thank you for pointing out though!

jennykim1016 commented 7 years ago

Before using the reweighting algorithm, the graph looks like the following. reweight before After using the reweighting algorithm, the graph looks like the following. reweight after

We need to check two things in this cornerplot :

  1. The blue histogram of redshift(z) looks like a gaussian distribution whose mean is similar as the red histogram.
  2. Because we are reweighting with respect to z, the shape of the other histograms beside the redshift one should not change.

I used both rejection sampling and general reweighting to do so, and I am pretty convinced that the current code is doing the right weighting. It might be good to check few more things - means and standard deviation of weighted lenses and quasars - but the z histogram looks promising.