Calibration of probabilistic predictions

[falcondai]

• The classic [DF83] paper and some of its classic results.
  • Well-calibratedness is weak.
  • Refinement.
• A frequentist criterion on beliefs.
• Assumptions:
  • Repeatability (it is the same state)
• A higher order reasoning?
• Recent developments.

[falcondai]

Connection to Bayesian reasoning.

• Repeatedly showing the same sample vs being IID.
• How does gradient descent "tell the difference"? Via regularization.