Closed otis-bath closed 2 years ago
Hi @brockfanning, just checking up on this one - not sure how demanding the changes required will be but looking at getting this into 1.7.0 if possible? Do you think this is realistic, wanting to finalise the release items for 1.7.0 next week. Thanks
@otis-bath The reason this is happening is that indicator 12.4.2 has both a "Units" column and a "Unit measure" column. The sdg-build is automatically converting "Units" to "UNIT_MEASURE" for the SDMX output, and then your custom mappings are also converting "Unit measure" to "UNIT_MEASURE". The end result is two columns with the same name, which causes that unhelpful error.
I think the fix is to automatically drop any existing columns that your custom mappings would duplicate. So, in the example described above, the "Units" column would be dropped (for the SDMX output). I'll work on a PR for that.
EDIT: And after fixing that, another issue presented itself - when there is a code-mapping for a column that contains "NaN" (empty values). I've attempted fixes for both issues in the same PR linked below.
@otis-bath #288 is ready for testing.
When making changes to a feature branch (feature-sdmx-test) to get the UK platform able to output SDMX data, the changes suggested to the config file are generating an error with the series. https://github.com/ONSdigital/sdg-data/runs/4396797341?check_suite_focus=true