search

global-electrification-platform / explorer

Global Electrification Platform (Web)
https://electrifynow.energydata.info/
MIT License
11 stars 7 forks source link

National clusters do not follow national boundaries #158

Closed bpstewar closed 5 years ago

bpstewar commented 5 years ago

I was perusing Malawi and I found a number of clusters that look like they fall outside the national boundary - this could be a problem. What are you using for the displayed national boundaries vs. the bounadries you used to create the clusters

image

babakkhavari commented 5 years ago

The administrative boundaries used for the clustering process are from GADM. They are slightly larger than the ones provided by OSM which seems to have been used on the platform.

bpstewar commented 5 years ago

So what can we do about it? The final product cannot have clusters crossing borders for national level analysis? Can you change the basemap to something that follows GADM boundaries? Could you try using the WBG basemap instead?

https://api.mapbox.com/styles/v1/gsdpm/cioenuwii001maznontujohu8.html?title=true&access_token=pk.eyJ1IjoiZ3NkcG0iLCJhIjoiY2lqbmN5eG9mMDBndHVmbTU5Mmg1djF6MiJ9.QqFCD7tcmccysN8GUClW8w#1.1/-0.000000/0.000000/0

https://datacatalog.worldbank.org/dataset/world-bank-basemap-collection (This second link might require special access to read)

nicolinalindblad commented 5 years ago

Pinging the importance of using the borders of the base layer mentioned by Ben above.

babakkhavari commented 5 years ago

Agree, I guess the first step would be to change the basemap used on the platform as this does not seem to correspond to the map sent by Ben neither, perhaps a change in basemap will solve the issue of clusters overlapping the borders. @vgeorge is it possible to change the basemap to the map that is linked by Ben?

babakkhavari commented 5 years ago

If a change in basemap does not fix the issue we will fix it on our side

vgeorge commented 5 years ago

The borders from the base map come from CIA World DataBank II. I've looked into GADM and Natural Earth, both seem less accurate than CIA's to me.

Natural Earth

Screenshot 2019-07-02 17 13 14

GADM

Screenshot 2019-07-02 17 13 50

It is possible to change the base map, but I believe those provided by Mapbox would display the same borders. I couldn't access look at the second link provided by @bpstewar. It must be publicly available to be used on the platform.

I believe this should be addressed via data side.

babakkhavari commented 5 years ago

@vgeorge and @bpstewar I have recieved the admin boundaries from the WB team. I will proceed to process the clusters with these borders so that they fit with the requirements from the WB-team.

akorkovelos commented 5 years ago

@vgeorge, our analysis was modified so the clusters are in alignment with the WBG basemap @bpstewar mentioned in a previous comment. Can we change the basemaps on the explorer accordingly?

vgeorge commented 5 years ago

@akorkovelos, as we discussed today, clusters ids should match featureId field in scenario files. If they don't, scenarios files should be updated an reingested. I would suggest waiting for the migration conclusion so we don't have to ingest the update files two times.

vgeorge commented 5 years ago

@akorkovelos @babakkhavari, I can't add the WBG basemap to the explorer. In order to do so we need it publicly accessible and get its mapbox id. Is the gsdpm Mapbox account owned by the WB? Is it posible to reach them to make the map public and provide its mapbox id? Another way would be to migrate that style and raw data to the Mapbox GEP account, but we would still need to reach them so we don't have to build the map from scratch.

akorkovelos commented 5 years ago

@nicolinalindblad can you please response in the absence of @bpstewar? Thanks.

vgeorge commented 5 years ago

@nicolinalindblad and @akorkovelos, you don't need to reach the authors anymore. with help from @danielfdsilva I was able to load the base map locally. It doesnt' show the clusters yet, I need to make some changes in order enable the them and will get back to you with the results.

vgeorge commented 5 years ago

I've managed to update the basemap with national borders provided by the WB-team, you can check at the live website:

The image below is from the same area reported by @bpstewar at the first comment:

image

@akorkovelos About the clusters files, in S3 exist two directories, Clusters with GADM and Cluster with WB admin, the second containing only Burundi. Should I update the clusters for all countries using GADM dir files or wait for the remaining countries to be available in WB admin dir?

akorkovelos commented 5 years ago

@vgeorge please wait for the rest of the countries in WB admin dir. We have just received HRSL for the last 4 remaining countries for which WB clusters are prepared now. We expect to finish and upload them all on S3 soon; not later than CoB tomorrow.

vgeorge commented 5 years ago

I believe this issue was addressed, please reopen if it is not the case.