Reliability and Calibrated Prediction

[NathanielF]

This will be an example notebook demonstrating the techniques of survival analysis used in reliability context with a focus on the predictive distribution and calibrated prediction intervals from a Bayesian and MLE perspective. Relates to issue: https://github.com/pymc-devs/pymc-examples/issues/474

Adding this draft version here, because i'm having a little trouble with the pre-commit checks

• [X] Notebook follows style guide https://docs.pymc.io/en/latest/contributing/jupyter_style.html
• [X] PR description contains a link to the relevant issue:
  • a tracker one for existing notebooks (tracker issues have the "tracker id" label)
  • or a proposal one for new notebooks
• [x] Check the notebook is not excluded from any pre-commit check: https://github.com/pymc-devs/pymc-examples/blob/main/.pre-commit-config.yaml

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[NathanielF]

Hi @drbenvincent and @OriolAbril

I think this is ready for review now. It's longer than i anticipated and i've cut out the idea to compare against conformal prediction as there is already quite a bit of detail in there.

The broader structure has three parts:

In the first part i'm just introducing time-to-failure data and the descriptive statistical analogues of survival and cdf functions. I follow a trajectory of first showing how we can analyse this kind of data with MLE or bootstrap style inference and then show on a trickier example with sparser data why the ability to add Bayesian priors to help calibrate the risk the profile is useful. Then i conclude with a "lets all be friends" kind of ending.

I'm not sure who is best placed to review, but since i deal with censored data and I know @drbenvincent recently updated the censored data notebook, i thought you might find it interesting? Curious about what you think too @OriolAbril? Happy to take any feedback on the content or structure...

[NathanielF]

Thanks so much for the review. @OriolAbril. I've quibbled with one of the requests but i think i've addressed the rest. I've added hide-input tags to a bunch of the data loads and some of the more involved plotting functions.

[NathanielF]

Thanks @OriolAbril I'm fine with all the changes you made, i understand the logic of the xarray-einstats version too. I think the code is fine, just using the dev version of xarray-einstats makes reproducibility harder. We could add a note to the effect that this uses a dev version of xarrray. What do you think?

[NathanielF]

Just fixing a typo at the beginning.

[NathanielF]

Woop, woop! Thanks 😊!