GMLAS mode: retrieve external codelists

BRGM / gml_application_schema_toolbox

GML Application Schema toolbox is a QGIS plugin allowing to manipulate OGC Complex Features

https://brgm.github.io/gml_application_schema_toolbox/

GNU General Public License v2.0

28 stars 18 forks source link

GMLAS mode: retrieve external codelists #38

Open sgrellet opened 7 years ago

sgrellet commented 7 years ago

Enable the user to do 'Resolve external' from the forms when DB structure was generated by the GMLAS driver mode

mhugo commented 7 years ago

@rouault Would calling GMLAS on the xlink href in "append" mode on the database work ?

If yes, how could the link between the existing layers and the xlink href be made ? Would it be handled by gdal or does it have to be handled application-side by the QGIS plugin ?

rouault commented 7 years ago

@rouault Would calling GMLAS on the xlink href in "append" mode on the database work ?

Probably in some (most ?) cases, but I don't think we can guarantee that in all cases, given the way the driver tries to generate unique layer names fitting in 64 characters (with truncation logic and addition of serial numbers when duplicates are found). When done on the same set of input schemas, this will of course always give the same result. But on another set of schemas, desambiguation might result in different layer names, for XML types that are identical.

If yes, how could the link between the existing layers and the xlink href be made ? Would it be handled by gdal or does it have to be handled application-side by the QGIS plugin ?

Should be done by QGIS. ogr2ogr append mode just appends. But you may also need to create new fields like parent_child_pkid if they don't exist initially.

mhugo commented 6 years ago

So, a possible scenario would be:

resolve the href link with GMLAS in a new database
merge it with the main database, ensuring there is no conflicts in layer names. copy metadata info in new layers (_href_ogr_fields_metadata for instance)
load new layers in QGIS
make QGIS aware of the links to the resolved href

sgrellet commented 6 years ago

+1 Does 'make QGIS aware of the links to the resolved href' means the previous href in the DB is replaced by the external key to the new content collected ?

mhugo commented 6 years ago

@sgrellet I'm not sure yet of the details, but I am leaning toward something non destructive. "Replacing" the external key would mean deleting "_href" fields and replacing them by a link to another layer. I think I will just add new fields and there will be a way to make the link to the new table (probably in metadata), but for now it will only be readable by QGIS

sgrellet commented 6 years ago

@mhugo : ok. In a way makes sense not to modify content added by the dataprovider

mhugo commented 6 years ago

Partially implemented in https://github.com/BRGM/gml_application_schema_toolbox/commit/9811d9e5a8cf0a2b0d1f11ef7019597f0cc3d21d

By "partially", I mean:

the href is resolved and new layers are created and linked to the current one
there is no way to update / delete an href link when it has been created

See #48 for the remaining work to do

sgrellet commented 6 years ago

non testable on v1.2.0-rc2 (linked to #53 bug report on the load layer button issue). note that the remote content will either be

codeList like http://inspire.ec.europa.eu/codelist/ResultAcquisitionSourceValue/inSitu/
other features like in 'http://ressource.brgm-rec.fr/data/Piezometre/02713X0132/PZ-8.2' you have a link to http://ressource.brgm-rec.fr/data/EntiteHydroGeol/230AA03

The load layer button approach as it is right now won't work for codeLists (no LoadLayer List capabilities). At least if the codeList repo is declared in gmlasconf.xml (ex : INSPIRE registry) it should be possible to propose another action (ex : retrieve codeList)

mhugo commented 6 years ago

@rouault About the loading of new codeList, I guess the only way to make GMLAS aware of new codeList is to add it to the gmlasconf.xml and reload everything ? Or is there a smarter way to do ?

rouault commented 6 years ago

@mhugo Yes if you declare a new codeList "repository" (ie a new URLSpecificResolution.URLPrefix in gmlasconf.xml parlance), you need to reload everything (or implement that behaviour on your side adding new fields, etc...)

sgrellet commented 6 years ago

Under GMLAS toolbox plugin : v1.2.0-rc7

Works fine with INSPIRE codeLists for example using https://forge.brgm.fr/svnrepository/epos/trunk/instances/BoreholeView.xml and http://ressource.brgm-rec.fr/data/Piezometre/06512X0037/STREMY.2 BUT when your dereference one codeList entry URI (ex : http://inspire.ec.europa.eu/codelist/BoreholePurposeValue/hydrogeologicalSurvey in BoreholeView.xml), the 'Load' button is not proposed for the other href anymore

Version de Python : 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] Version de QGIS : 3.0.1-Girona Girona, a86bec25eb

sgrellet commented 6 years ago

We will need to be able to configure in the Plugin settings what format (http Accept) should be used with which URI. I know it kinds of mimics what GMLAS conf does (see gmlasconf-inspire.xml). But that's also needed in XML mode -> will create a special issue for this.