Statistical Significance Flag or Indicator

[AmberSJones]

We need to implement some sort of flag to indicate whether values are significant or not. This would be calculated on the fly using a fairly simple chi-square statistic.

[Maurier]

Can you give me an example of this?

[AmberSJones]

We forgot to discuss this today (1/22). I think we'd need to get with Doug or Courtney to see an example calculation. At this point, the sample sizes are not large enough for statistical significance.

[AmberSJones]

Update 4/8: When Doug sends the new data file, he'll provide an example of a determination of statistical significance. We will then need to implement on-the-fly calculation for cross-dimensional questions (i.e., a 5x5 matrix of responses). In these cases, we'll implement the flag as a positive or negative warning sign in the bottom left corner (intersection of x and y axes).

[Maurier]

In addition to this feature, the heatmap now uses a cutoff of 5 respondents as requested in our last meeting.

[Maurier]

The flag has been implemented.

[AmberSJones]

Ok- I want to do a few checks on this... there are a few cases where the result was not significant that I was curious about.

Also, I wonder if we should put the text next to the flag icon to make it more obvious since the hovering is slow.

[AmberSJones]

Ok, I did some spot calculations, which verify the result that you're showing.

What do you think about putting text next to the flag? We might need input from @horsburgh and Doug.

[horsburgh]

So - there is a single flag that tells me whether the whole matrix is significant? I'm not sure how to interpret the flag. If it is a single flag for the whole table, I think you should just put the text next to the flag in the same box. Plus, it would be good if the text was a link to a pop up or something that has a paragraph describing the test that is used to determine whether it is significant or not.

[AmberSJones]

Yes, a single flag for the whole matrix to say whether the results of that matrix are significant or not. I think both of your suggestions are good. Maybe Doug can compose the text- or I could try to extract it from his email.

[Maurier]

The flag now has the text next to it instead of on hover. I changed the style a little as well. Let me know what text should be displayed on click.

[AmberSJones]

How's this for some text: 'This flag indicates whether the differences in the results in this matrix are statistically significant according to a chi-square test. The distribution of observed frequencies is compared to expected frequencies from a distribution where there is no relationship between the two axes. If the total difference is greater than the chi-square statistic for the 95% confidence level, than the results are considered statistically significant.'

[Maurier]

The text has been added and the pop up is triggered when clicking on the flag text.

[AmberSJones]

Works for me. Closing.