Add new Protected and Conservation Areas layers

[jeremyshelton]

@dimasciput @NyakudyaA we need to add new Protected and Conservation Areas layers, and remove the Protected Areas layer currently on FBIS (because I is outdated).

The layers can be accessed here: https://egis.environment.gov.za/gis_data_downloads

Can we please include the above URL link in the layer description.

Thanks

[NyakudyaA]

@amyburness Can you tackle this since you you assigned with tasks here

[amyburness]

@NyakudyaA Yes I'll look at it today.

[amyburness]

@jeremyshelton @NyakudyaA Hi there.I have been to the site and gotten all the data available, but it is not similar to the protected areas that are on FBIS. It appears to be mainly biosphere areas etc. I'm not sure if I am missing something.

[amyburness]

here is a sample of the data.

[jeremyshelton]

Hi @amyburness Thanks for getting onto this so quickly. There should be a variety of sublayers with the protected and conservation areas as per:

Here is their interactive map that shows the layers spatially: https://dffeportal.environment.gov.za/portal/apps/webappviewer/index.html?id=7e27f116dd194c1f9d446dacc76fe483

I think we probably need to remove the existing protected areas stuff from the CBA tabs (found under spatial filters on FBIS), and create a new tab called Protected areas with sub-drawers Protected areas a (9 types that can be turned on and off, default all on) and Conservation areas (3 types that can be turned on and off, default all on).

Ideally, these should be available as both spatial filters and spatial layers.

@helendallas @NyakudyaA

[amyburness]

@jeremyshelton Ok let me have a deeper look

[jeremyshelton]

OK cool @amyburness Just shout if you want to chat...

[dimasciput]

@amyburness any updates on this ticket?

[ThiashaV]

@jeremyshelton

Conservation areas:

Protected areas:

[jeremyshelton]

Thank you @ThiashaV

This is looking good.

Have both layers ben adde to the FBIS testing site? (testing.healthyrivers.kartoza.com) ....I can see the Protected areas layer, but not the Conservation Areas layer.

We would also like to add these as a spatial filters in a new tao called "Protected Area":

Protected Areas: Protected Area Conservation Area

Happy to jump on a call and discuss if needed.

[jeremyshelton]

@timlinux @dimasciput apparently DFFE update these layers fairly regularly. Is there a way to automate that updates travel through to FBIS periodically, or would that be a huge mission?

[NyakudyaA]

@jeremyshelton Unfortunately they do not provide WMS endpoints that we can plug into. I tried to access https://dffeportal.environment.gov.za/hosting/rest/services/PACA/SACAD/MapServer from QGIS but it seems the layer is not complete. If we can get that endpoint working then we can use it directly in GeoServer

[NyakudyaA]

I will add the layers to testing and Geocontect as well

[NyakudyaA]

@jeremyshelton We now have access to the WMS endpoints and can pull the layers directly into our map gallery. However due to the limitation of the server ArcGIS or https://gis.stackexchange.com/questions/57526/geoserver-cascading-wms-featureinfo-is-lost once the layers are in our instance users cannot click on a feature and get a description of what that feature is. See the example below:

[jeremyshelton]

Hhmmm OK thanks @NyakudyaA - I appreciate your work on this. Do you know if the layer descriptions (protected/conservation area names) will come through when csv files are downloaded? If not, can you think of a possible workaround for this issue? @helendallas

[NyakudyaA]

If we publish them ourselves, then we can get the attributes etc when clicking on a feature.

Though in Geocontext we could try to use the direct connections from them(their URL) and we should return all the latest values to use in the spatial filter. Need to test this.

The layer overlays will have to be manually updated quarterly when they publish new versions.

[jeremyshelton]

OK copy that. Thanks @NyakudyaA Please let us know once you have tested the geocontext option and then we can decide. @helendallas

[amyburness]

@jeremyshelton do you want all of the protected area's layers including things like heritage sights and botanical gardens available as per the image above/coppied below?

[jeremyshelton]

@amyburness yes please, if possible. @NyakudyaA have you had a chance to test the geocontext layer connections yet?

[amyburness]

@jeremyshelton the geocontext appears to be working I just need to add all the protected area types but that sould be relatively quick. @NyakudyaA your thoughts?

[NyakudyaA]

@amyburness So you will need to publish all those layers in our geoserver and then in geocontect make sure that they use the direct connection. Please note that the URL will change in future for direct connection to geocontext so maybe you might need ti add a note so that you can remember this.

[jeremyshelton]

@amyburness @NyakudyaA sounds good to me. Let's go for it.

[NyakudyaA]

@jeremyshelton An alternative to this would be https://github.com/kartoza/django-bims/issues/3379. This will allow ArcGIS layers to be used directly as Non-Biodiversity layers. Currently Geocontext already supports using ArcGIS layers directly

[helendallas]

Please can we decide on approach and move forward.

[jeremyshelton]

@NyakudyaA @amyburness @timlinux @dimasciput @helendallas

Can we please have a conversation to resolve this approach?

I suggest we do this on Friday as part of the sprint planning call if that works for everyone?

Thanks

[helendallas]

@amyburness @NyakudyaA Was this actioned during the last sprint?

[helendallas]

@amyburness @NyakudyaA Please update us on this. We have another sprint from tomorrow

[amyburness]

@helendallas @admire and I think we have a solution for this.

[NyakudyaA]

@amyburness I have added the first layer in the DB, please publish it through GeoServer. Here is the logic to fetch the layer from that URL

CREATE SERVER enviroment_affairs
    FOREIGN DATA WRAPPER ogr_fdw
    OPTIONS (
        datasource 'WFS:https://dffeportal.environment.gov.za/hosting/services/PACA/SAPAD/MapServer/WFSServer?request=GetCapabilities',
        format 'WFS' );

    CREATE FOREIGN TABLE sanbi.protected_area_wfs (
        OBJECTID bigint,
        WDPAID int,
        CUR_NME varchar,
        WMCM_TYPE varchar,
        MAJ_TYPE varchar,
        SITE_TYPE varchar,
        D_DCLAR timestamp,
        LEGAL_STAT varchar,
        GIS_AREA double precision,
        PROC_DOC varchar,
        Shape_STArea double precision,
        Shape_STLength double precision,
        geom geometry(MultiPolygon, 4326)
    )
    SERVER "enviroment_affairs"
    OPTIONS (layer 'ENV_DB.LDR.Protected_Environment');

create materialized view sanbi.mv_protected_area_wfs as 
select * from sanbi.protected_area_wfs;

[amyburness]

Thanks @NyakudyaA, @helen all of the layers are now being fetched as WFS I publish to Geoserver and FBIS ASAP.

[jeremyshelton]

Thanks @amyburness Are you ready for us to test this now?

[helendallas]

@amyburness Please action this please

[helendallas]

@amyburness The layer is in FBIS. Has the automation being actioned to keep it updated as per ticket. Please respond ASAP and confirm if I can close this ticket.

[amyburness]

@helendallas and @jeremyshelton the New PA layer is up on the Layer selector.

For the filter:

1. The current filter is for all of the protected areas by name? should this remain and just be updated?
2. I would like to add a filter per protected area type, i.e. have a draw for Forest Nature Reserve and another for Forest Wilderness etc that will then open to select the parks by name.

is this logic correct?

[helendallas]

Thanks @amyburness

Re 1. Yes we need them by name - Filter as is: Protected Area. But suggest renaming Protected Area Name It may also be useful to add by Protected Area Type. I don't think create new drawers though as Kate and I have spent a lot of time fixing a rearranging these.

Rather include user current filter

• Protected Area Name
• Protected Area Type Make sense?

[jeremyshelton]

@amyburness shout it you would like to set up a quick call if clarity on this is needed?

[amyburness]

hey there I would are you free in the next few mins

On Wed, 18 Oct 2023 at 09:19, Jeremy Shelton @.***> wrote:

@amyburness https://github.com/amyburness shout it you would like to set up a quick call if clarity on this is needed?

— Reply to this email directly, view it on GitHub https://github.com/kartoza/django-bims/issues/3324#issuecomment-1767820951, or unsubscribe https://github.com/notifications/unsubscribe-auth/ASR4I2U72HCX4CTDYHPYEJTX757I3AVCNFSM6AAAAAAVFSNIESVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRXHAZDAOJVGE . You are receiving this because you were mentioned.Message ID: @.***>

--

Amy Ternent GIS Technician Visit http://kartoza.com to find out about open source:

• Desktop GIS programming services
• Geospatial web development
• GIS Training
• Consulting Services Office: +27(0)71 225 0039

[amyburness]

@helendallas the new layers are in the layer selector and I have added them to the filters. I just need to finalize the numbering and ask Dimas to harvest them.

[helendallas]

Thanks @amyburness Not sure why your FBIS filters are different to mine??? You have protected areas and conservation areas, still in the old NFEPA draw....all this has changed. I assume you will integrate into current spatial filter drawers?

[dimasciput]

@helendallas that was layer selector panel in Amy's screenshot.

[amyburness]

@helendallas they are in the correct draw and numbered as follows. let me know if this is correct?

[helendallas]

ah okay thanks @dimasciput Please then harvest so we can finalise,

[helendallas]

looks good, thanks @amyburness

[amyburness]

So these layers are made from data pulled directly from the Department of Environmental Affairs server. However, if there is an update to the data an admin will have to trigger a reharvest of the data to the materialised view tables.

I tested the filters for 1 representative point and it is working so @dimasciput you can harvest for all.

[helendallas]

Thanks, @dimasciput We proudly need functionality to allow an admin person to trigger a harvest rather than always having to ask you.

[amyburness]

You have access to the DB so I can make some docs on how to do it using some SQL.

[NyakudyaA]

@amyburness How do we know the layers have been updated? If it's a known cycle i.e 2 times a year we could automatically schedule updating the materialized views.

Please note that there is a magic function in the DB that you could call like

select kartoza.refreshallmaterializedviews('sanbi');

This will refresh all materialized views in the sanbi schema.

Other things to consider

• When EVGIS update the layers, do they only add more data or change the structure of the layer i.e. add new columns etc? If it's the latter then we will need to recreate the foreign tables, recreate the MViews, update the GeoServer configuration for the layers and reharvest.

[amyburness]

@NyakudyaA I can't see if there is a known cycle or what attribute that have, looking at https://egis.environment.gov.za/data_egis/data_download/current now to check.

[helendallas]

@amyburness Only one has come through for each filter....see below. Please check urgently . we have a conference and training on Monday.

[amyburness]

@helendallas I just tested on 1 point @dimasciput is harvesting all the others in the background.