Make raw candy power rankings data available

[aaronrudkin]

Your candy power rankings data has the wrong unit of analysis: your initial study matched Candy A against Candy B, Candy C, Candy D, etc. But your published research aggregates all of Candy A into a single row. As a result, your unit of observation is a candy (instead of a candy-candy pairing), your N is equal to the number of distinct candies, and you have no power. You also have an issue where your DVs are nonindependent. You have N=85 when you should have as many as N=7,140 observations if you tested every distinct pairing, and actually you should have more than that, because there should be more than one vote per pairing, so really the unit of analysis should be "vote on a candy-candy pairing". The consequences of the lack of power are serious, maybe even fatal to your analysis.

You discuss your R^2, which is fine (although Adj R^2 = 0.45 is hardly surprising given how many features you had), but you never actually address the model more generally. Although the model is jointly significant, almost all of the features you tested are not even remotely significant. Even setting aside NHST as a statistical objective, in a holistic sense, if we interpret the full confidence intervals of the feature parameters, we are left with a muddy and unclear story.

I know the point of the piece is we're supposed to laugh because only old people like hard candy, but your standard errors are huge. I don't even have confidence that hard candy is bad? What kind of world are we living in?

I suspect some of this is coming from VIF thanks to collinearity between certain features, and some of this is probably coming from certain features being fairly rare in the dataset. But the source of both these problems is that you're testing the wrong unit of analysis.

This is an important issue, maybe the most important 538 has ever investigated -- certainly more important than that article last week where you just let partisan hacks yell about how their team was winning and presented that as journalism... real pulitzer material there -- but it is not being treated with the seriousness it deserves.

Please release your raw candy-candy matchup data so candy scholars can leverage it to generate better research.

[Lex7777]

Если честно, я вообще не понимаю о чем речь идёт!?

[Pr0methean]

Another benefit of publishing the pairwise data is that we'd be able to find the Condorcet winner!

[aaronrudkin]

Just wanted to acknowledge that it's been a year without movement on this crucial issue, and the original candy power rankings article was republished with nary an attempt to address it :(