Autodiscover OWS services available in GeoNode instance

[cezio]

Typical GHC workflow assumes that user adds one OWS at once. However, GIS applications can serve data in different way, with several different OWS endpoints available. In this case, GeoNode connects several OGC services (sometimes from different physical services). If user would want to monitor external availability of GeoNode instance with GHC, it would be a significant effort to add all available OWS endpoints.

For this, i'd like to add simplified way to add GeoNode instance to monitoring: GN can list used OWS endpoints with https://github.com/GeoNode/geonode/issues/3250, and GHC can pick it and automatically populate proper resources for each endpoint available. Integration would work in following way:

• user adds GeoNode resource type and provides homepage of GeoNode instance as url
• GHC internally queries autodiscovery api
• by default, all resources created in this way will have one common tag: GeoNode {hostname}.
• for each OWS found, separate Resource is created (with tags)
• invalid OWS endpoints (like returning 404) will be discarded, and only working ones will be added.

[justb4]

Approach looks good. Some remarks on scenario:

• invalid OWS endpoints : GHC still adds them, the Resource service may be temporarily down and we've seen cases where a WMS only supports 1.3.0. This can be edited in the default assigned WMSCapabilitesProbe for that OWS.
• what about deletion? If you delete GeoNode Resource, you expect all its OWS Resources to be deleted?
• will some GeoNode API endpoints require basic auth? There is an issue #9 for that.
• you may consider assigning default GeoNodeAPIProbe for the GeoNode API Resource healthcheck

Also briefly looked at first commit. Think approach in general ok, need to look in more detail:

• the sniff_test_resource will need more abstraction, your "jump table" is a good start. Thinking of even make ResourceTypes and their handlers Plugins. As a next step after integration.
• the Plugin code that queries the GN API for OWSs would belong to the GN ResourceType handler, not in the plugin.probe module.

As said, a first quick scan of the commit, need more time for reviewing details.

[cezio]

Hi,

Thanks for comments. Sorry for delayed answer.

• invalid OWS endpoints : GHC still adds them, the Resource service may be temporarily down and we've seen cases where a WMS only supports 1.3.0. This can be edited in the default assigned WMSCapabilitesProbe for that OWS.

Good point. I'll restore adding resources, even if they failed with sniffer.

• what about deletion? If you delete GeoNode Resource, you expect all its OWS Resources to be deleted?

That's the problem for which I don't see any reasonable solution at the moment. there's no linkage between resources yet, afair. Grouping resources (#18) could be helpful here. Just to be clear about this: I don't create any master-resource for GeoNode, just add resources for each url returned in ows endpoints list. However, with some implementation for resource grouping may it may be a good revisit this.

will some GeoNode API endpoints require basic auth?

I'd say this depends on GN deployment configuration. This endpoint should be handled by unmanned clients, so either basic auth or token would work here. At this point i guess this can be passed as a part of url, as you suggest in referred issue.

you may consider assigning default GeoNodeAPIProbe for the GeoNode API Resource healthcheck

Yes, I think this is something to consider in next steps. Since GN has resources monitoring built-in, metrics could be checked remotely with GHC too. But as I wrote, at this moment this integration just adds endpoints returned by GN, nothing more.

the sniff_test_resource will need more abstraction, your "jump table" is a good start. Thinking of even make ResourceTypes and their handlers Plugins. As a next step after integration.

True, it could be better. Setuptools' entry points feature could be used here, so each ResourceType can be registered (and allow to include code from external packages)

[justb4]

Ok, thanks for your patience @cezio . PR #160 now merged into dev branch. All tests ok up to now. Only cannot test GeoNode instance, since IMO it requires the /api/ows_endpoints/ API. I tested e.g. http://geonode.state.gov, though the URL itself was registered, no OWS services. Is this the intended behaviour? Or should we fail registering a GeoNode instance that does not support or fails the /api/ows_endpoints/ API? You may want to test on http://dev.geohealthcheck.org/ where the latest dev branch runs.

It will still be useful and quite trivial to define (default) Probes for a GeoNode instance API endpoint itself like e.g. api/layers etc. This would be similar to Probes for SensorThings API. No specific code is even required, just configuring some Checks. Something for a separate issue.

[cezio]

thanks!

Only cannot test GeoNode instance, since IMO it requires the /api/ows_endpoints/ API.

Yes, autodiscovery requires either latest GeoNode (supporting code is in master, I'm not sure if it's in any recent release yet), or to install https://pypi.python.org/pypi/geonode-ows-endpoints for older installations.

[justb4]

We can close this one, has been stable, will be in v0.3.0 coming soon. @cezio thanks for your work (and patience). Pls open additional issues for any maintenance or a GeoNode API Probe.