mysociety / theyworkforyou

Keeping tabs on the UK's parliaments and assemblies
http://www.theyworkforyou.com/
Other
224 stars 51 forks source link

Consider removing Flesch-Kincaid metric #1078

Open wfdd opened 8 years ago

wfdd commented 8 years ago

The Flesch-Kincaid tests have increasingly come to be used by media outlets to rank the complexity of speech of politicians among themselves and through time; and, time and again, they've been criticised by linguists and others as plain lacking scientific grounding and not really being indicative of anything, at all. For example:

dracos commented 8 years ago

We do have an entire FAQ saying that most of the numbers in this section are pointless, linked to from the section: http://www.theyworkforyou.com/help/#numbers . Most obviously, Hansard is not what people actually said, it is an edited transcript. You might want to use it within its own dataset to see how MPs might differ in their use of language, but no, it isn't indicative of anything.

wfdd commented 8 years ago

The statistics are not pointless; they merely lack context. The FK score is a special case in that it draws a wrong conclusion. It's one thing to state a fact, e.g. 'Mary has spoken in 42 debates this year', and it is another thing entirely to say, Mary speaks like a 15-year-old 'cause there's more short words in a sample of her vocabulary.