CPR 2020 data: use Pacific region for comparative assessment

[tmfrnz]

from @annemariebrook via email

the comparative assessment for the CPRs for the Pacific countries can now be calculated relative to a Pacific region reference group (the first region of the world where we can do this!)

[tmfrnz]

@annemariebrook @kchadclay could you clarify how to handle HI countries?

Currently for the CPR comparison, the HI countries are compared with other HI countries only, and other countries with all countries.

For ESR, HI countries are compared with other HIs, other non-HI countries are compared with all other countries in the same region (or sub-region for Pacific)

Now the question is, if the Pacific comparison should likewise only be enabled for non-HI countries or also for HI countries?

[tmfrnz]

To further clarify: CPR comp. assessment is either for HI and OECD countries or for all countries

[annemariebrook]

@tmfrnz, thanks for the question. The Pacific comparison should be enabled for the countries in the Pacific that are both non-HI and non-OECD. i.e. all countries except NZ and Australia.

HI and OECD countries should continue to be compared only with other HI and OECD countries.

Thank you!

[annemariebrook]

Another clarification: the reference group for the Pacific region should include ALL countries in the region. i.e. Fiji's Empowerment score will be compared to a regional average that includes NZ and Australia.

[tmfrnz]

To recap: while we always do a regional (or sub-regional) comparison for ESR, for CPR we will have 3 different types of comparisons:

1. HI/OECD: compared with all High Income/OECD countries, incl self
2. Not HI/not OECD: compared with all other countries
3. Region/not HI/not OECD): compared with all other countries in region (ie Pacific)

For each type we may/will need to cover the following cases

1. No data
2. Score for dimension (all rights)
3. ~Score for 1 right only~
4. ~Score for multiple rights (but not all)~

EDIT: only offer for complete data (see comment below)

[tmfrnz]

This is what we currently have:

1.1./2.1 No data

[Safety from the State] data have not yet been produced for [Austria]. We would like to expand our data collection to the whole world as soon as possible. To learn more about how you can help, please visit this page on the HRMI website.

1.2. HI/OECD // all rights

Compared with the small group of five (?) high-income OECD countries we have civil and political rights data for, [New Zealand] is performing close to average [on the right to be safe from the state | on empowerment rights]

Note: we should use variable for number of HI/OECD countries

2.2. Not HI/OECD // all rights

For the civil and political rights we do not have data for enough countries in [Sub-Saharan Africa] to do a regional comparison. However, compared to the other [18] countries in our sample, [Angola] is performing close to average [on the right to be safe from the state | on empowerment rights]

---

A few observations:

• The comparisons both include references to the reference group and the metric
• This is similar as for ESR, however the order is different: first we refer to the metric and then to the region (eg On Quality of Life, Angola is performing worse than average, compared with the other countries in Sub-Saharan Africa). The same is true for comparisons of incomplete data (1 or multiple rights, eg On Quality of Life, when we look across the rights for which we have data, Austria is performing close to average, compared with all other high income countries)

---

Suggestions for implementation

Now for maximum re-use we should strive to structure the new sentences in a way that parts can be re-used, ideally following the order of the ESR example.

First part: what metric(s)

~1 right~

~On [Safety from the State], [Vanuatu] is performing worse than average on the [Right to freedom from extrajudicial execution], ...~

EDIT: only offer for complete data (see comment below)

~Some rights~

~On [Safety from the State], when we look across the rights for which we have data, [Vanuatu] is performing close to average, ...~

EDIT: after all it might actually not be sensible to calculate the rights averages here, see https://github.com/HumanRightsMeasurementInitiative/hrmi-dataportal/issues/143#issuecomment-626844550

EDIT: only offer for complete data (see comment below)

Dimension (all rights)

I would suggest to re-order the sentences where we have a dimension score, as this will (a) not require different treatment of the 2 different dimensions and (b) make it easier to combine with reference group part.

_On [Safety from the State] Angola is performing worse than average, ...

Second part: what reference group

HI/OECD

... compared with the small group of [five] high-income OECD countries we have data for.

Note: again should use variable for number of HI/OECD countries

Region (Pacific

... compared with other countries in the [Pacific] we have data for.

Other (not Pacific, not HI/OECD)

... compared with other countries that we have data for.

Question: should there be any additional information on the lack of regional data? (see 2.2. above)

For example:

... compared with other countries that we have data for (we do not have data for enough countries in [Sub-Saharan Africa] to do a regional comparison).

[annemariebrook]

Thanks Timo. As noted in issue #143, we propose NOT to include a Safety from the State or Empowerment comparative assessment if we don't have a complete set of Safety from the State or Empowerment scores.

So that means your recap above is correct with the one addition that the comparison will only be made for countries where all scores in a CPR dimension are available.

For countries where there is at least one missing CPR score, for the relevant CPR dimension/s we will instead include the new proposed text as proposed in issue #143.

[tmfrnz]

@annemariebrook @kchadclay further questions:

For adding the regional comparison should we stick with (a) the current logical sentence order OR (b) the reverse order (as currently used for ESR)?

a) Compared with the other countries in the [Pacific] we have civil and political rights data for, [Tuvalu] is performing close to average [on the right to be safe from the state | on empowerment rights]

b) On [Safety from the State] [Tuvalu] is performing close to average, compared with other countries in the [Pacific] we have data for.

---

If going with (b) I would probably also suggest aligning the other texts with the ESR order:

Not HI/OECD, no regional comparison:

On [Safety from the State] Angola is performing worse than average compared with other countries that we have data for (we do not have data for enough countries in [Sub-Saharan Africa] to do a regional comparison).

Instead of current:

For the civil and political rights we do not have data for enough countries in [Sub-Saharan Africa] to do a regional comparison. However, compared to the other [18] countries in our sample, [Angola] is performing close to average [on the right to be safe from the state | on empowerment rights]

HI/OECD

On [Safety from the State] [New Zealand] is performing close to average compared with the small group of [five] high-income OECD countries we have data for.

Instead of current:

Compared with the small group of five (?) high-income OECD countries we have civil and political rights data for, [New Zealand] is performing close to average [on the right to be safe from the state | on empowerment rights]

---

Please advise

[annemariebrook]

@tmfrnz, I have a fairly strong preference for the sentences starting with "Compared with...". In order to make them consistent (which is a good idea) I would rather rephrase the Quality of Life Sentences. e.g.

Compared with all other high income countries, the United States is performing worse than average on Quality of Life rights (this comparison is calculated using the 'Income adjusted' benchmark).

Instead of current:

On Quality of Life, the United States is performing worse than average, compared with all other high income countries (this comparison is calculated using the 'Income adjusted' benchmark).

[tmfrnz]

Great - thanks for confirming!

[tmfrnz]

@annemariebrook could you quickly confirm that for CPR data the comparative assessment is only using the subregion for countries from the "Pacific" and not all subregions (as we do for ESR)?

[tmfrnz]

@annemariebrook deployed to v3.1 site for you to review (CPR comparing with subregion only for Pacific countries)

As the sentence structure has been harmonised (ESR now like CPR: "Compared with ... [country] performs ... on [dimension/rights]"), please make sure to also review and update all translations (branch "cpr-comp-assessment-141", fields: "hrmi.components.CountryNarrative.compAssessment[XXX]")

[annemariebrook]

@annemariebrook could you quickly confirm that for CPR data the comparative assessment is only using the subregion for countries from the "Pacific" and not all subregions (as we do for ESR)?

Correct (as the Pacific is the only region where we have enough CPR data), thanks @tmfrnz.

[annemariebrook]

@annemariebrook deployed to v3.1 site for you to review (CPR comparing with subregion only for Pacific countries)

As the sentence structure has been harmonised (ESR now like CPR: "Compared with ... [country] performs ... on [dimension/rights]"), please make sure to also review and update all translations (branch "cpr-comp-assessment-141", fields: "hrmi.components.CountryNarrative.compAssessment[XXX]")

This looks good to me. Thanks @tmfrnz. I will pass the translation on to Sarah.

[tmfrnz]

@sarahbichan, this has now been merged into the v3.1 branch so feel free to review and edit the translations there

[annemariebrook]

Hello @tmfrnz and @sarahbichan. I think @sarahbichan told me she has loaded the translations for these fields. However, they are not showing up on v3.1. e.g. for Samoa I see this: . Keen to get these finalised ASAP, thanks both!

[tmfrnz]

@annemariebrook, @sarahbichan it appears that there are 2 issues with the ES translation:

1. The first part should refer to the reference group of countries (EN: "Compared with the other countries in the Pacific..." - this was caused by the variable "referenceCount" still present in the ES translation - now fixed e600001c4357f557aa9bec4f06c03c21a499594d
2. The 2nd part simply requires removing the "TODO" I think but would leave it up to you to review and update (field "hrmi.components.CountryNarrative.compAssessmentESR.endAll", currently L324)

[tmfrnz]

@sarahbichan currently the total number of "TODOS" in the language files is

• 9 in ES
• 8 in FR
• 9 in PT

Please review and update these fields

[sarahbichan]

The TODO highlighted in point 2 above has already been removed, but I will check for any remaining TODOs now.

[sarahbichan]

I removed/reviewed the TODOs in Atom and it seemed like it let me commit the changes. Looking at it in GitHub, they are still there. I will redo this work in GitHub now to ensure it goes through properly. Not sure why it didn't work in Atom, sorry.