Some clusters at the border of state are highlighted even when no settlements are within the filter selection (extreme value for distance to grid for example).

[Bachibouzouk]

A problem remains with filtered clusters: some clusters at the border of state are highlighted even when no settlements are within the filter selection (extreme value for distance to grid for example).

To reproduce the bug:

• refresh the webmap
• select Sokoto
• click on the funnel symbol for remotely mapped settlements to expand the options
• move lower bound slider of "distance to grid" to the far right (to filter out all the settlements) --> 3 settlements are still visible on the map (ids: 40658, 40523 and 39302), west and south from the main city in the state

Interesting facts:

• Switching to the neighboring states and they appear there as well
• Zooming in dehighlight them and seems to fix the issue, I still could not understand where it came from

Originally posted by @Bachibouzouk in https://github.com/rl-institut/NESP2/pull/333#issuecomment-722308323

[christian-rli]

It took me a while, but I found the reason for this. There are two clusters over each other. They have the same id. The filter function will change one of the clusters because it filters over every id and assigns a different style to every id. Leaflet won't think to do the same thing to the same id twice.

Finding out why there are two clusters sometimes was difficult. The database appears to store the some clusters, with the same cluster_id twice - namely clusters that are at a border. The clusters will be listed once for every adjacent postcode. Is this correct? At least it appears to me this way. At any rate the update script that creates the tiles uses filters by state and not by postcode. It was my understanding that these two should align perfectly.

To fix the issue there are at least two ways. the database can either drop double entries or the update script needs to filter via postcode (i.e. if postcodes do uniquely reference clusters).

I meant to change the update script to do that, but it seems I get an error when running this script locally now: sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) permission denied for materialized view cluster_offgrid_mv

The change shouldn't be too difficult, but I also don't really want to do it blindly... It would take me unnecessarily long, I think. @AWeissMotz I think you implemented the script on the server (I was unable to find it there). After the In the followings line(s) there just needs to be an additional filter that drops clusters in states, if the listed postcode doesn't match the postcode of the state. That way no two clusters with the same id get written to the geojsons and eventually the tileserver.

for offgrid clusters: https://github.com/rl-institut/NESP2/blob/f517ae2afba7b62d93105bb732442b0c455839b6/server_updates/update_data_from_database.py#L154

for the other clusters: https://github.com/rl-institut/NESP2/blob/f517ae2afba7b62d93105bb732442b0c455839b6/server_updates/update_data_from_database.py#L211

Do you think you can implement this @AWeissMotz ? If no, please direct me to the script on the server or maybe if you can explain any difference in how to access the database so I can fix it in the update script. As I have take over all tasks from a collague during his parental leave I hardly have any time left, but I may find some in the evening.

[christian-rli]

The issue seems to have been fixed on the server side. When downloading clusters they only appear once and with one postcode. Thank you for the fix! The issue remains however for the calculated clusters. @AWeissMotz you may do the same thing in the database there and then close this issue.