martinapugliese
commented
7 years ago Also, for style:
- [x] the plot title is not in LateX
- [x] the references part: not all orange
Open martinapugliese opened 7 years ago
martinapugliese
commented
7 years ago Also, for style:
Identify a PL graphically
Pareto quantile-quantile plots (Q-Q plots), or log-log plots.
Q-Q plot (fare voce separata)
[from wikipedia]
means plotting the quantiles of two distribs one against the other. It is a parametric plot with parameter being the number (index) of the quantile. If the two distribs are similar, points will be on the diagonal. If they are linearly related, points will be on a line (not only the diagonal). Typically better than comparing histograms. P-P plot is the same thing but plots cumulative distrib functions.
For PLs (Pareto QQ plots), plot the quantiles of the log-transformed data against the quantiles of an exponential dist with mean 1
log-log plot
See if it's a line, but only good when there is lots of data
Estimate the exponent
MLE
see wikipedia
KS
PL and lognorm2
Plotting
log -log scale and logarithmic binning, see Newman review cumulative plotting (called frequency/rank), see Newman review. Cumulative equals to frequency/rank because
Zipf and Pareto plot the cumulative and one plots y vs x, the other the reverse, see Newman review.
finding the exponent: crap to OLS, see formulas Newman review (and proof, it is MLE)
Few real world distribs follow a whole PL, most have a PL tali (see newman examples, try to find data and reproduce?). It adds difficulty in trying to find the exponent because one needs to know where to cut
also see http://www.hpl.hp.com/research/idl/papers/ranking/ranking.html (cited by Newman)
maths of pow laws, see Newman again, see partivculry the discussion on the infinite mean, the distrib of wealth (80/20 rule, Lorenz curves)
Yule process for speciation, from Newman
Generating pow laws
[follows Newman]
combination of exps. See the example on word lenght and the how Shannon formulated his theory [deserves a separate chapter]
riche gets richer (aka preferential attachment) see Newman but expand?
Also see appendices in Newman
See and follow recipe pag 3 powlaw-empirical Newman paper
see in particular the distribution fit comparisons on the powerlaw paper
from same paper, note this " the central limit theorem. When random variables are summed, the result is the normal distribu- tion. However, when positive random variables are multiplied, the result is the lognormal distribution, which is quite heavy-tailed.", in generative mechs (read deeplypero')
definition Newman cumulative and wikipedia definition: ???
From plos python paper
"when presented on log axes should use logarithmic binning": refer to blog post on loglog
log binning means exponentially increasing bin widhts
this is because altough linear binning have high resolution in the entire range, the small prob of observing high values prevents computation of reliable prob there. Log binning inreases the likelihood of observing values in the tail, compensated by normalising with larger width All this^ deserves a blog post per se
pacchetto powerlaw uses log bining by default
the xmin discussion
the goodness of fit (in compare distributions)
exp being the minimum alternative for evaluating heavy-tailedness