The data update workflow does not write some data with special characters to the csv data file, resulting in some data being omitted on the data hub.
For example, data for Métis is omitted for indicators 4.1.1/5.2.1 due to the é in Métis. Non-breaking spaces are also known to cause this issue (see 4.1.1 where the space between Inuk and Inuit is non-breaking) as well as EN dashes (see 16.1.1).
This issue occurs only when the scripts are run via GitHub Actions. It does not occur when scripts are run directly on local systems.
The data update workflow does not write some data with special characters to the csv data file, resulting in some data being omitted on the data hub.
For example, data for Métis is omitted for indicators 4.1.1/5.2.1 due to the é in Métis. Non-breaking spaces are also known to cause this issue (see 4.1.1 where the space between Inuk and Inuit is non-breaking) as well as EN dashes (see 16.1.1).
This issue occurs only when the scripts are run via GitHub Actions. It does not occur when scripts are run directly on local systems.