We may need to reconsider how we compute stddev in our graphs

[plschmitz]

For the topics in a facing, and for facing averages, we are computing the stddev of all questions subsumed by the context. This has much more variation that the variation in the average coverage of the focus (facing or topic), and likely misleads folks looking at the data.

We should probably compute the set of averages for each thing we are graphing, and then compute the stddev of that set of averages. This is more work in the code, but will likely produce more sensible results to folks who are not steeped in the data.

[plschmitz]

See somewhat related issue #158

[plschmitz]

I did the following test: averages = {'researcher':[], 'data':[], 'systems':[], 'software':[], 'strategy':[], } stddevs = {'researcher':[], 'data':[], 'systems':[], 'software':[], 'strategy':[], } assessments = CapabilitiesAssessment.objects.filter(review_status=CapabilitiesAssessment.ReviewStatusChoices.APPROVED) for ass in assessments: answersByFacing = ass.filtered_answers().group_by_facing() for facing in answersByFacing: answers = answersByFacing[facing] agg = answers.aggregate_score() averages[facing.slug].append(agg["average"]) stddevs[facing.slug].append(agg["stddev"])

for facing in Facing.objects.all():
    nInsts = len(averages[facing.slug])
    avgOfAvgs = sum(averages[facing.slug])/nInsts
    avgOfStds = sum(stddevs[facing.slug])/nInsts
    s = pd.Series(averages[facing.slug])
    stdofAvgs = s.std()
    print(f'{facing}: average of averages: {avgOfAvgs}  average of std: {avgOfStds} std of Averags: {stdofAvgs}')

This yielded the results: researcher: average of averages: 0.5884650735294116 average of std: 0.20250959856004808 std of Averags: 0.20220817854751216 data: average of averages: 0.5295393318965519 average of std: 0.24755299041699946 std of Averags: 0.20316877016596363 software: average of averages: 0.480896875 average of std: 0.3041595971763341 std of Averags: 0.19115443626232947 systems: average of averages: 0.6103618421052632 average of std: 0.2831038208124409 std of Averags: 0.1844905095151738 strategy: average of averages: 0.6161686197916665 average of std: 0.22439221732661338 std of Averags: 0.21309750021524237

The data we generate for the summary capmodeldata is: Facings Average Values Std Dev 0 Researcher-
Facing 0.588557 0.292187 1 Data-
Facing 0.535691 0.325348 2 Software-
Facing 0.482134 0.369173 3 System-
Facing 0.607973 0.343485 4 Strategy & Policy-
Facing 0.613416 0.309139

Note that the average of standard deviations is quite a bit smaller than the stddev across all the answers, and the std of averages is even smaller (often about 2/3 of the stddev across all the answers).

I have no idea how to do the Stddev of the computed averages across institutions for each facing, in the QuerySet world.

[plschmitz]

Moving to 2.2 as this will be non-trivial.

[plschmitz]

To look at for 2.1.5