search

CamDavidsonPilon / Probabilistic-Programming-and-Bayesian-Methods-for-Hackers

aka "Bayesian Methods for Hackers": An introduction to Bayesian methods + probabilistic programming with a computation/understanding-first, mathematics-second point of view. All in pure Python ;)
http://camdavidsonpilon.github.io/Probabilistic-Programming-and-Bayesian-Methods-for-Hackers/
MIT License
26.76k stars 7.87k forks source link

About a statement in Chapter 1 #232

Open aaarbk opened 9 years ago

aaarbk commented 9 years ago

First, let me start by saying this is an amazing effort! I look forward to working through the whole book.

My comment is on the statement: "Graphically, a probability distribution is a curve where the probability of an outcome is proportional to the height of the curve." in chapter 1, section "Probability Distributions".

This statement would be somewhat correct if we were talking about the probability distribution of a discrete random variable - except in that case the distribution would just be a list of values, rather than a curve. In case of a continuous distribution, the height of the curve doesn't carry such a straightforward interpretation, rather it is the area under the curve for an interval p1 < x < p2 that is equal to the probability of the random variable taking on a value between p1 and p2.

Intuitively it might not seem unreasonable to say that the probability of an outcome is proportional to the height of a curve (even after you accept the area interpretation), I think we run the risk of facing inconsistencies in the mathematics involved, and ultimately our understanding, if we take that notion to heart.

For real-valued random variables, while you can concretely talk about the probability of x taking on a value in a neighbourhood of some number x0, making the neighbourhood as (finitely) small as you like (eg. "what is the probability that x has a value between 10 ± 0.0001?") the probability of x having a value of exactly 10 (or any other number) is zero.

Anyway, sorry about the pedantry, and keep up the great work!

alexgarel commented 9 years ago

I would say that this kind of correction could be added as a footnote. Giving notions that speaks to the reader is some times more important than being exact, but a footnote doesn't hurt.

CamDavidsonPilon commented 9 years ago

Ah, I had a response but didn't hit Comment and it was lost. I'll type it again:

Probably the most common misunderstanding in probability distributions is this exact thing: pdf's can exceed 1. It make sense from a mathematical POV, but when you're learning statistics and probability theory and probably your mathematics is not strong, it can be confusing. For the majority of readers, likely they fall into this category.

I do have a note somewhere in the text (and now I can't find it!) that links to this question: http://stats.stackexchange.com/questions/4220/a-probability-distribution-value-exceeding-1-is-ok

I really like this answer, and I could not explain it better myself - hence the link and not an inline description.

Overall, it's wrong, but it's 90% correct too, sort of like the Bohr model of an atom: incorrect, but a good starting ground =)

pieteradejong commented 8 years ago

@CamDavidsonPilon would it make sense if I inserted a sentence or footnote like below, right after "You can see examples in the first figure of this chapter."? Is there another place in the book where pdf's are discussed in more detail, where that link might make more sense?

"For a more detailed explanation of the difference between the height of the curve, and the area below the curve, see this link."