search

open-connectome-classes / StatConn-Spring-2015-Info

introductory material
18 stars 4 forks source link

Evaluation of Clustering Algorithms #170

Open indigorose1 opened 9 years ago

indigorose1 commented 9 years ago

I've been having trouble distinguishing between the different evaluation methods used in the paper from last lecture: ICC, ARI, and AIC. Does anyone know of any resources that compare these methods (and possibly others) so that their efficacy is easier to understand? It's been exceptionally hard trying to find user-friendly information on these.

imichaelnorris commented 9 years ago

I don't know of any good sources comparing them.

The Wikipedia article for AIC has comparisons with other selection methods like the Bayesian information criterion. The ICC article on Wikipedia doesn't compare it to anything else.

I find the AIC article on Wikipedia to be decently descriptive. Do you find the Wikipedia articles unhelpful? Maybe a better description of them could be written

Edit: misread efficacy for efficiency....

kristinmg commented 9 years ago

I think the AIC score was used to compare the methods (they used it to evaluate if the differences between the partitions were significant), whereas the other two were used to evaluate the quality of each partition. Because the three solutions come from 3 different methods it's challenging to compare them statistically. The ARI score can quantify the level of similarity between partitions of interest, but it cannot infer on whether one partition has significantly better fit than another partition. But using the ICC and AIC criterion they were able to compare qualitatively all 3 estimates. The ICC score tells us how homogeneous some biological feature is within the partitions of a proposed network decomposition, and then the AIC score is used to assess if the differences between the partitions from the three methods were significant.