heidimok
commented
3 months ago Hi @RicardoGomesIST, this was one of the open questions for the municipal data. We were wondering where thecodigo value on the spreadsheet comes from as we don’t see that field in the spatial data (municipalities_gadm41_PRT_2.zip). Would you be able to comment on it? Feel free to ask any questions here as well as a comment and the team can clarify.
cc @alukach - please also comment if this open question is no longer relevant as I know things can change fast as we continue to make progress.
RicardoGomesIST
yellowcap
Note: This is a Slack discussion (from @alukach) I'm bringing into GitHub so we can track ongoing municipal data questions and updates as comments here for better communication.
Context
We received the latest municipal data (v5) from Ricardo. Municipal Data v5 - Google Sheets
Some outstanding issues with the data:
The following rows in the
metricssheet are missing corresponding geometries in the municipal spatial data:3101, 3103, 3102, 4801, 4401, 4601, 4502, 4301, 4901, 3110, 3104, 4501, 3108, 4603, 3106, 3107, 3201, 4701, 4302, 4802, 3109, 4602, 3105Additionally, the Study sheet has a few issues. Some field names needed to be update (some of these changes to requirements are new, not Tecnico's fault in any way) and the fields that are being used to join Geometries to Metrics are incorrect. Anthony was able to get the ingestion working by making the following changes to the study sheet:
A corrected sheet can be found here: https://docs.google.com/spreadsheets/d/1Zwoohp8Zq8D4LgLY1aIrB5LTLRvxmV9dr8FAghphCV0/edit#gid=1485987722
Data Issues
Here is the output from the seed operation (ie ingesting the geometries & metrics). Joining on the
NAME_2field of the geometry is somewhat problematic in that there duplicate geometries that share that field in the geospatial data. If a duplicate is found in the geometry, we log + ignore and continue. If a geometry is found without a corresponding metric, we log + ignore and continue.Reading the above logs, we have a geometry with
NAME_2=Praia da Vitóriathat has no metric. We also have a metric withnombre=Vila da Praia da Vitóriawith no geometry. It seems reasonable that the metric row should be updated to trimVila dafrom the name. As per https://github.com/developmentseed/tecnico-energy-app/issues/26, we drop any metrics that don’t have geometries.The core issue is that we’re joining the geometry data with the metrics data on a non-ideal field.
Questions
Understand from Ricardo where the
codigovalue on the spreadsheet comes from as we don’t see that field in the spatial data (municipalities_gadm41_PRT_2.zip).We have two separate sets of geometries with duplicate NAME_2 properties:
LagoaandCalheta. ForLagoa, there appears to be a town in the Azores, and then a city on the south coast. ForCalheta, there appears to a region on the southeastern half of Sao Jorge island and another on the souther half of the western side of Madeira island. So these are definitely separate communities.See the screenshot of one of the
Calhetageometry properties, the metric data showscodigo=3101for Calheta but we don’t have any property like that.Other Notes
@alukach highlighted data that we added in the spreadsheet in yellow and any columns/keys that we needed to rename in fuchsia
