search

diana-hep / carl

Likelihood-free inference toolbox.
BSD 3-Clause "New" or "Revised" License
57 stars 22 forks source link

[MRG] adding InterpolatedIsotonicRegression #35

Closed cranmer closed 8 years ago

cranmer commented 8 years ago

not sure if this is the right place to put this class, or how to modify CalibratedClassifierCV to use it.

cranmer commented 8 years ago

Some updates to the InterpolatedIsotonicRegression class and a failed attempt to integrate it into the calibration

glouppe commented 8 years ago

Could you add this new method to the tests? It should be limited to adding "interpolated-isotonic" to https://github.com/diana-hep/carl/blob/master/tests/learning/test_calibration.py#L78

glouppe commented 8 years ago

Not critical, but to enforce good coding practices throughout the codebase, could you check PEP8 and PEP257 issues? (you can run flake8 to check these)

cranmer commented 8 years ago

ok, basically working now. Here are some results: https://github.com/cranmer/carl-notebooks/blob/iso-interp/Approximating%20likelihood%20ratios%20with%20calibrated%20classifiers.ipynb

Need to understand some of the tests in Travis. Not sure what is up with Brier score. I notice in my test that both isotonic and the interpolated one sometimes violate the contract of predict_proba. Also, not sure if the test is consistent with predict_proba[:,1]

glouppe commented 8 years ago

Merging this, thanks for the added calibration method!