owid / owid-grapher

A platform for creating interactive data visualizations
https://ourworldindata.org
MIT License
1.4k stars 228 forks source link

Data: new continent indicator #4176

Open lucasrodes opened 1 week ago

lucasrodes commented 1 week ago

Context

I've updated our charts using our continent indicator (variableId 123) to use an equivalent one which is in ETL (variable id 900801). This indicator is based on our regions.yml definitions!

More context in https://github.com/owid/owid-issues/issues/1738.

In order to complete this migration, there some additional steps needed (continue reading...).

Next steps

1) Colors

Currently, the colors used for each of the continents are not correct (in scatter plots, Marimekkos, etc.). Instead, we should use our defaults.

Note: This also affects explorer views.

2) Citation

We should not add an explicit citation to the source of this indicator in our charts (this is the behaviour that we previously had with variable 123).

In addition, I would suggest that we never show the source in chart footers for indicators in dataset regions/2023-01-01/regions (dataset id 6497). That's because we may want to color countries by other standards (e.g. WB, WHO, etc.).

The image below shows a chart with issues 1 and 2:

Image

3) Data points shown in gray

I've detected some scatter plots, where the data is shown in gray (regardless of the continent):

Image

4) Data is not found

I couldn't migrate the following charts. If I attempt to change variable 123 for the new indicator, I get a "No matching data" message and no data is shown. Therefore, I've decided not to migrate these.

Image Image

danyx23 commented 1 week ago

I put a PR up that fixes a simple subset of the problems described here. For issues 3 and 4 maybe part of the issue is that Antarctica is no longer part of the mix in the new var? Just a vague idea, might be something entirely else.

veronikasamborska1994 commented 6 days ago

@danyx23 I don't think it's likely Antarctica is the issue for 3 and 4 - I've checked and if you recreate the same chart from scratch it doesn't seem to be causing an issue - for example this chart. I don't know what's exactly the issue but the problem seems to appear when you set the timeline max (see the screenshot where I've replicated it on the new draft chart that otherwise looks fine until you set the maximum on the timeline). I've briefly looked at the charts in 3 and 4 and seems like this might be the unifying feature across them

Image

veronikasamborska1994 commented 6 days ago

@danyx23 we can fix all of these by setting the timeline max to default and simply using tolerance for some of these charts as I think the main reason why they were truncated was to show the year where more data was available - e.g., here is a fix for https://ourworldindata.org/grapher/distribution-democracy-index-polity chart Image

lucasrodes commented 6 days ago

In case this is helpful, here you can compare the old and the new continent indicators (left: new, right: old):

Image

New indicator reports data for 2023, and the old for 2015.

danyx23 commented 3 days ago

@veronikasamborska1994 ah thank you, that is very helpful! This means the error occurs because the new regions indicator has 2023 as the year and the old continents indciator had all the regions as 2015. Color should have an infinite tolerance implicitly but apparently there is a bug there somewhere.