tmfrnz
commented
5 years ago To make it less likely that users draw wrong conclusions, several options have been discussed:
1. Use and visualise qualitative grades instead of scores for ESR metrics in summary view
Rather than visualising scores on a continuous scale, grades would be shown on a discrete 6 step scale, emphasising the difference between ESR and CPR
Concerns:
- it is uncertain if stakeholders support qualitative grading
- still users may visually compare ESR and CPR
- representation will be inconsistent with all other representation of ESR scores (overview, metric view, country drilldown which can be confusing for the user
- data shown will be inconsistent with CPR data (and users may wonder why not also use qualitative grades for CPR data)
2. Show (not visualise qualitative grades) instead of visualising scores for ESR metrics in summary view
Rather than visualising any data, grades would be displayed (letters), emphasising the difference between ESR and CPR
Concerns:
- it is uncertain if stakeholders support qualitative grading
- not showing any visual representation will be inconsistent with all other displays of ESR scores (overview, metric view, country drilldown which can be confusing for the user
- data shown will be inconsistent with CPR data (and users may wonder why not also use qualitative grades for CPR data)
- looks very awkward on dimension level (just one number)
3. Introduce qualitative grades for ESR and CPR in addition to score visualisation
This will allow comparing performance while keeping a consistent representation of performance scores across the site
Concerns:
- it is uncertain if stakeholders support qualitative grading
- users may be confused about different grading scales (could appear like error in app)
4. Show uncertainty bars for CPR data
Showing uncertainty bars will make the CPR data visualisation different from the ESR visualisation, also making the representation more consistent with metric and country drill-downs (but a little less so for the overview page diamonds).
Concerns:
- Users could still visually compare scores but it may be less likely.
- Some users may struggle to interpret uncertainty bars
5. Emphasise ESR gap
Visually emphasising the achievement gap (still keeping horizontal bars) will make the ESR data visualisation different from the CPR visualisation - this could be applied to all other representation of ESR data. Users could however still visually compare scores but it may be less likely.
Concerns:
- Users could still visually compare scores but it may be less likely.
- Users may mis-interpret scores and confuse gap with score, especially ehwn displayed next to CPR scores
- Will likely not work on overview
- This may have colour implications
6. Split country report (narrative) into ESR and CPR tabs
Splitting ESR and CPR reports into 2 tabs will make it impossible to compare scores on country page. Will no longer need summary and need to split people at risk table and comparative assessment.
Concerns
- Users may miss the 2nd report tab
annemariebrook
When looking at country aggregates, users may compare bar lengths of ESR and CPR values and infer that country A is doing better or worse for ESR then for CPR if the bar is longer. However the 2 types of scores are on different scales using different methodologies and should thus not be compared. E.g if a country scores 88% for ESR (immediate duty) and 8.1 of 10 for Empowerment, it does not mean that its performance for ESR is better than for Empowerment.