Differences in estimation with a big dataset

[lsaravia]

I am fitting power laws with a big (10^6) data set, I suspected that there was some problem with alpha, so I tried the pareto.R functions from Shalizi http://tuvalu.santafe.edu/~aaronc/powerlaws/, and it gives me different results

the code I used is here

https://github.com/lsaravia/CriticalGlobalForest/blob/master/R/test_poweRlaw.r

[csgillespie]

Thanks for the feedback.

• Could I get a copy of the date (simulated or otherwise)
• Your url doesn't work.

Cheers

[lsaravia]

My mistake, I have used the function pareto.fit instead of zeta.fit, after correct that the results are identical. Anyway I wonder why if you look at the figure, the continuous power law seems to fit better.

The code is here

https://www.dropbox.com/sh/91ywo3x510z4ozk/AABafDJY9GQ9BM-sVDsNaXQca?dl=0

and the data

https://www.dropbox.com/s/u6zq51mqhnkqxiy/MOD44B.MRTWEB.A2010065.005.Percent_Tree_Cover.tif.bin?dl=0

Cheers!

[csgillespie]

Sorry for the delay. I responded a few days ago, but must have forgotten hit comment.

Anyway, I've noticed in types of data, the Clausett et al method doesn't work that well; it's the downside for having a nice flexible method.

I'm working on other methods, would it be possible to include you dataset in the paper as an example?

Thanks

[lsaravia]

Yes you could, but I guess that the problem is that the data is originally continuous (tree patch size), and it is discretized by the remote sensing image, so the continuous function fits better. Soon I will deposite the data in a public repository, in the meantime you could use it.

[csgillespie]

Great. Do you have a paper I could cite?

[lsaravia]

Not yet... but I will let you know

[csgillespie]

As an aside, there is a bug in http://tuvalu.santafe.edu/~aaronc/powerlaws/plfit.r when the data is discrete and xmin=1.