search

aimalz / qp

Quantile Parametrization for probability distribution functions module
MIT License
3 stars 9 forks source link

Bug in quantile interpolation #83

Closed aimalz closed 7 years ago

aimalz commented 7 years ago

There's a bug in the reconstruction of the PDF from quantiles that's causing them to blow up at the endpoints. I'm going to change the interpolation function for quantiles to require endpoints at which the CDF is 0 and 1 so it doesn't cause nonsensical linear interpolation. This will require changing the syntax for qp.PDF.quantize(). I'm going to use this opportunity to make the syntax more consistent as was suggested in #79.

aimalz commented 7 years ago

The bug is resolved and merged into the paper branch via two tricks. The first trick was to choose endpoints such that linear interpolation still satisfied the definition of the quantiles (duh). The second trick was to start the optimizer with a good guess of where the quantiles were by evaluating the CDF and finding the closest places to the desired quantiles. Now quantization is faster and more accurate, an improvement that propagates all the way to the final results!

drphilmarshall commented 7 years ago

Excellent - nice work! Looking fwd to seeing the plots - good luck, and have fun!

aimalz commented 7 years ago

This fix doesn't exactly deserve praise. Though it guarantees a solid solution for the quantiles, it still has issues. For example, it broke the nosetests because it takes too long on perverse cases where the first guess of the quantiles is terrible.