search

OneGeology / documentation

OneGeology Documentation
https://onegeology.github.io/documentation/
Creative Commons Attribution Share Alike 4.0 International
3 stars 1 forks source link

Discoverability of exemplar datasets #6

Open nmtoken opened 3 years ago

nmtoken commented 3 years ago

I knew they existed, but finding the page where they are mentioned is buried. Need to make more obvious links to content at:

ftp://ftp.bgs.ac.uk/pubload/OneGeology/

nmtoken commented 3 years ago

On the cookbook content currently listed on the OneGeology website through:

Is hierarchy, though Cookbook example data is shown in top listing.

But it misses: http://onegeology.org/service_provision/server_setup/mapserver.html#configuring-the-exemplar-services which links to the shapefiles and image used by the MapServer example.

===

In OneGeology documentation linked to from the Portal (https://onegeology.github.io/documentation/index.html)

Navigate to

There's no obvious link to any exemplar datasets.

Expanding Using GeoServer section in the menu leads to: https://onegeology.github.io/documentation/providingdata.html#example-data

AND

Expanding Using MapServer lets us discover:

https://onegeology.github.io/documentation/providingdata.html#configuring-the-exemplar-services (Configuring the exemplar services) which also has exemplar (and different data).

===

Looking at the shapefile data:

shapefileunharmonised.zip which is found in the GeoServer data section contains bedrock625ll.shp which is the same data as bedrock625ll.shp in 1G_WMS-exemplar-data-MS6-update.zip. However shapefileunharmonised.zip does not include superficial625ll.shp.

Looking at the raster:

georeferencedimage.zip which is found in the GeoServer data section contains bedrock625ll.png which is the same data as bedrock625ll.png in 1G_WMS-exemplar-data-MS6-update.zip. georeferencedimage.zip also contains a legend.

=== Looking at the geopackage data (GeologicUnitView.gpkg), this is the same geometry as bedrock625ll.shp, but attribution is different.

Sveouu commented 3 years ago

Looking at the shapefile data:

shapefileunharmonised.zip which is found in the GeoServer data section contains bedrock625ll.shp which is the same data as bedrock625ll.shp in 1G_WMS-exemplar-data-MS6-update.zip. However shapefileunharmonised.zip does not include superficial625ll.shp.

Yes, I noticed that too, but that being said I believe neither of the example data files directly mirrors the example used under https://onegeology.github.io/documentation/providingdata.html#using-arcgis When working through the ArcGIS server example. the shapefile attribution used in the example is different. I could not find a similar attribution in the G_WMS-exemplar-data-MS6-update.zip or in the shapefile (shapefile unharmonized) example data. It might just be a small issue but it could throw users like it did with me.

Trouble finding the links to this ftp server: --> ftp://ftp.bgs.ac.uk/pubload/OneGeology/ ftp server is not referenced in the cookbook on GitHub, but only on the cookbook on the website http://onegeology.org/service_provision/server_setup/exampledata.html The ftp server does not open in Chrome. --> use Internet Explorer/ edge. -->may be issues with local chrome settings.

Issues when using the wrong dataset to publish: I initially used the GeoPackage file. --> before serving the data, the data needs to be in a filegeodb or geodatabase

Hints for publishing data: Data source not registered with the Server, allow copying data to the server?

When publishing a service, ensure that the .mxd and data source of the layers to publish is set to the UNC path as recognized by your Arc GIS server directory.

i.e UNC path = "//kwsan/workspace/ …" is equal to "W:/ …", but the UNC specification needs to be used. If the UNC path is not specified the data will be copied to the server.

INSPIRE compliance Achieving INSPIRE compliance with the steps outlined may need to be reviewed slightly. I have had a look at using FME to run checks & ensure datasets are INSPIRE compliant. This may not be necessary for OneGeology as we cannot expect all Users/Members having access to FME. But it could possibly be used for other projects within BGS where INSPIRE compliance is required.

to be continued...

nmtoken commented 3 years ago

There are three aspects to consider for INSPIRE compliance, one is linking to appropriate metadata, in the extended capabilities section of the Capabilities response (either by reference to a metadata record, or by embedding the data), the second is the provision of in scope data as a download service (SOS,WFS,WCS... but not WMS), according to the relevant data model, the third is the provision of a view of the data (for in scope data) using the mandated styles (where applicable, though often styles are only recommended.) As part of provision of view and download services there is a requirement to use the specified layer names.

FME can run as a service, so perhaps we could add a validation service to test compliance, though there is also an INSPIRE validator, that should also be able to do it.

FME can I think be used to create an INSPIRE dataset so we could document how to do this, though ArcGIS for INSPIRE is meant to offer a similar functionality. One issue with ArcGIS for INSPIRE is that for Geology it doesn't use GeoSciML. If we can show how to use FME to create a GeoSciML 4 based ArcGIS service, that would be great.

Another contender here is HALE.

bgsmase commented 3 years ago

Possibly trying to maintain a unified source of data sets for each of the different bits of server software (MapServer, GeoServer and ESRI) is too ambitious as it requires checking any changes in example data against each bit of software. Also they don't all deal with the same kinds of data source. So maybe better to let each server software section refer to separate sources of example source data sets.

bgsmase commented 2 years ago

FTP is gradually becoming less usable, not supported by browsers, awkward to get through firewalls, so we need to make exemplar content available elsewhere.

nmtoken commented 2 years ago

Is github suitable?

bgsmase commented 2 years ago

GitHub is suitable for configuration files especially as it will enable us to version control them easily. It might not be suitable for larger data files. There are notes at https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github about this. Installing Git LFS is a bit of an extra step. It explicitly says GitHub is not suitable for large SQL files. Need to check the sizes of our data files.

nmtoken commented 2 years ago

Individual zips (data, or data+software) are in the range 10's of MB, largest zip is 104MB. Largest file is TIFF c. 453MB