mapfishapp & geonetwork - export current map context as a "map metadata"

[fvanderbiest]

SV_EDITOR or SV_REVIEWER or SV_ADMIN users should be able to send their current map context to geonetwork, in order to generate a new "dynamic map" metadata.

As of now, users will be able to fill the following information in mapfishapp:

• title
• abstract

The following ones will be automatically set:

• data sources: from the layer capabilities
• author: from the current logged in user
• extent: from WMC
• projection: from WMC
• WMC url (eg: http://sdi.georchestra.org/mapfishapp/ws/wmc/geodocf24dcba462994398742ccc1e291b49ae.wmc )
• date

We could POST to a dedicated geonetwork service the relevant information (either the full WMC string, or the WMC URL, or all of the above fields in a custom JSON string, to be discussed)

The resulting metadata should provide a link to the original WMC and an other one to a viewing URL (something like XXX/mapfishapp?wmc=http://YYY)

Once the data has been processed, the geonetwork service should return the metadata URL. mapfishapp will then open this metadata in a new browser tab.

[bchartier]

It would be nice to have a boolean flag in the map contexts table indicating which contexts are referenced in the catalogue (in order to prevent removal of old contexts referenced in the catalogue by a database administrator).

[fvanderbiest]

Agreed. So, on geonetwork response received, we should toggle this boolean flag.

[pmauduit]

There already is a XSL stylesheet to translate WMC into metadata:

https://github.com/geonetwork/core-geonetwork/blob/develop/web/src/main/webapp/xsl/conversion/import/OGCWMC-to-ISO19139.xsl

The work on this ticket will be from my side to:

• implement a new jeeves service onto regular geonetwork (stable-develop)
• propose a PR to integrate the service
• backport to geOrchestra's geonetwork

[pmauduit]

It would be nice to have a boolean flag in the map contexts table indicating which contexts are referenced in the catalogue (in order to prevent removal of old contexts referenced in the catalogue by a database administrator).

Why not a metadata ID / UUID or null if not referenced, instead of a boolean ?

[pmauduit]

It is also possible to use / adapt the existing xml.mef.import.ui and improve the current stylesheet, because not so much information are extracted from the wmc as for now.

[fxprunayre]

Do not hesitate to improve the current stylesheet if needed. A metadata on map should look like this template : https://github.com/geonetwork/core-geonetwork/blob/develop/schemas/iso19139/src/main/plugin/iso19139/templates/map.xml

It's probably better to have a custom service rather than changing xml.mef.import.ui because you'll have to add input paramters (eg. title, abstract, ...) populated in mapfishapp ?

[bchartier]

It would be nice to have a boolean flag in the map contexts table indicating which contexts are referenced in the catalogue (in order to prevent removal of old contexts referenced in the catalogue by a database administrator).

Why not a metadata ID / UUID or null if not referenced, instead of a boolean ?

From my point of view, a flag can be of any type. Boolean, UUID, whatever...

[fvanderbiest]

Why not a metadata ID / UUID or null if not referenced, instead of a boolean ?

Better idea, true.

[fvanderbiest]

It's probably better to have a custom service rather than changing xml.mef.import.ui because you'll have to add input paramters (eg. title, abstract, ...) populated in mapfishapp ?

The WMC string should contain all the information about the map, including title and abstract... but not about the author.

The author name could maybe be obtained on the geonetwork side ?

[fvanderbiest]

Side note, mainly for @bchartier.
geOrchestra 14.12 should be able to serialize map contexts into OGC OWSContexts, thus taking into account WMTS layers. But the current improvement only targets OGC WMC documents.

[hsquividant]

+1 ! OWS Context can also deliver a set of configured WPS

[fvanderbiest]

A UI-related question. I'm planning to add a checkbox in the lower left corner of this window dialog, stating "save map to catalog".

Everyone's OK with this proposal ?

[fvanderbiest]

Or maybe better : a new button in the same window, "save to catalog" ?

[fvanderbiest]

preliminary work done in branch issue751

[bchartier]

Or maybe better : a new button in the same window, "save to catalog" ?

+1

[fphg]

+1

[fxprunayre]

Sum-up discussion we had today with @fvanderbiest and @pmauduit

Improve XSLT making WMC to ISO19139

https://github.com/geonetwork/core-geonetwork/blob/develop/web/src/main/webapp/xsl/conversion/import/OGCWMC-to-ISO19139.xsl

data sources: from the layer capabilities

2 types of information could be used to link the MDM (metadata on map) and the MDD (metadata on data sets) used in the map.

In the WMC, we may have MetadataURL (only one per layer):

 <LayerList>
    <Layer ....
      <MetadataURL>
        <OnlineResource xlink:type="simple" xlink:href="http://www.cgiar-csi.org/data/srtm-90m-digital-elevation-database-v4-1"/>
      </MetadataURL>

This should be added to the dataQuality section as a source (this was the strategy adopted for EEA internal catalog of maps):

-- If the URL is a GeoNetwork like types of URL, extract the UUID form the URL and set the @uuidref + the xlink:href

<gmd:LI_Lineage>
    <gmd:source uuidref="f888cfb0-88fd-11da-a88f-000d939bc5d8" xlink:href=”https://sdi.eea.europa.eu/internal-catalogue/srv/eng/csw?SERVICE=CSW&VERSION=2.0.2&REQUEST=GetRecordById&ID=8c74e8c4-701a-4cd1-a988-7f2bf9e891bd”>
    </gmd:source>
</gmd:LI_Lineage>

-- If the URL is not containing any UUID
<gmd:LI_Lineage>
    <gmd:source xlink:href=”http://www.cgiar-csi.org/data/srtm-90m-digital-elevation-database-v4-1”>
    </gmd:source>
</gmd:LI_Lineage>

We may have ol:attribution

<ol:attribution xmlns:ol="http://openlayers.org/context">
          <Title>geOrchestra PSC</Title>
          <OnlineResource xlink:type="simple" xlink:href="http://sdi.georchestra.org/"/>
          <LogoURL width="196" height="50" format="image/png">
            <OnlineResource xlink:type="simple" xlink:href="http://sdi.georchestra.org/header/img/logo.png"/>
          </LogoURL>
        </ol:attribution>
      </Extension>

In that case, add to the identificationInfo/credit the list of layers and their attributions:

* <LayerName> (<Extension/ol:attribution>)

author: from the current logged in user

In the XSLT, call a Java function (eg. https://github.com/geonetwork/core-geonetwork/blob/develop/core/src/main/java/org/fao/geonet/util/XslUtil.java#L272) returning the user session information and create a proper CI_ResponsibleParty as metadata/contact and metadata/identificationInfo/pointOfContact. The Java function could return a DOM object to easily manipulate it in the XSLT.

extent: from WMC

Done in the current XSLT

projection: from WMC

Done in the current XSLT

WMC url (eg: http://sdi.georchestra.org/mapfishapp/ws/wmc/geodocf24dcba462994398742ccc1e291b49ae.wmc )

This needs to be a parameter sent to the service which creates the metadata record and added as an online resource element (see template map.xml).

date

Current date time I would say eg. https://github.com/geonetwork/core-geonetwork/blob/develop/web/src/main/webapp/xsl/conversion/OGCWxSGetCapabilitiesto19119/identification.xsl#L57

Make a new Jeeves services

Being relevant for other GN users, I would recommend to make the implementation on GeoNetwork/stable-develop and backport it to geOrchestra's GN. Create a new jeeves service to make the backport easy (ie. not a SpringMVC service - which will be the default for GN 3 service). The client side part (made in mapfishapp) could be done at a later stage in GN Angular mapviewer (ongoing work).

[fvanderbiest]

Many thanks for the hints Fx :-)

[fvanderbiest]

The client side (in branch issue751) currently POSTs a JSON string to the GN controller, which is as follows:

{
    "wmc_string": "<?xml version=\"1.0\" encoding=\"UTF-8\"?><ViewContext .... ",
    "wmc_url": "http://xx/mapfishapp/ws/wmc/geodocecd1fdb3bedd4c958b6b69a52a2e3ad5.wmc", 
    "viewer_url": "http://xx/mapfishapp/?wmc=http%3A%2F%2Fxx%2Fmapfishapp%2Fws%2Fwmc%2Fgeodocecd1fdb3bedd4c958b6b69a52a2e3ad5.wmc"
}

If this is not what geonetwork should expect, please tell me and I'll adjust the parameters accordingly.

Be warned that the wmc_string might be invalid XML (eg when generated with Chrome 37), see https://github.com/georchestra/georchestra/issues/759. It should be validated GN-side.

wmc_url is a link to the file containing the wmc_string, hosted by mapfishapp (forever, stored in db). It should be recorded in the generated metadata on map (MDM).

viewer_url is a permalink to the mapfishapp viewing service for the given context. It should also be stored in the generated MDM.

[fvanderbiest]

Be warned that the wmc_string might be invalid XML (eg when generated with Chrome 37), see #759. It should be validated GN-side.

Though this should be fixed from 13.09 on with the workaround introduced in https://github.com/georchestra/georchestra/commit/ed6dcb089a763012fdf984b69900cdd0a631e4e4

[fvanderbiest]

The wmc_url value should IMO be stored as /gmd:MD_Metadata/gmd:distributionInfo/gmd:MD_Distribution/gmd:transferOptions/gmd:MD_DigitalTransferOptions/gmd:onLine/gmd:CI_OnlineResource/gmd:linkage/gmd:URL with gmd:CI_OnlineResource/gmd:protocol set to OGC:WMC-1.1.0

[fvanderbiest]

I have no good idea where to store the viewer_url information. /gmd:MD_Metadata/gmd:distributionInfo/gmd:MD_Distribution/gmd:transferOptions too ?

Maybe @fxprunayre ?

[fxprunayre]

I have no good idea where to store the viewer_url information.

I would say, same place gmd:onLine/gmd:CI_OnlineResource/gmd:linkage/gmd:URL but with protocol URL link to simply have an hyperlink from the catalog results to open the mapfishapp with the WMC.

[fvanderbiest]

Sounds good. Thanks again !

[pmauduit]

{ "wmc_string": "...", "wmc_url": "http://xx/mapfishapp/ws/wmc/geodocecd1fdb3bedd4c958b6b69a52a2e3ad5.wmc", "viewer_url": "http://xx/mapfishapp/?.." }

It would actually be better / more Jeeves-compliant to have classic data encoded values posted, e.g.

wmc_string=...&wmc_url=...&viewer_url=...

[fvanderbiest]

OK, thanks. Will modify the client accordingly before the end of the week.

[pmauduit]

As a side note, I planned the entry point to be /geonetwork/srv/[lang]/wmc.import

[fvanderbiest]

@pmauduit : fixed both of your remarks in branch issue751

[pmauduit]

State:

• Mapfishapp-side: fixed a problem with the POST on the GeoNetwork webservice: needed to be x-www-form-data/urlencoded : https://github.com/georchestra/georchestra/commit/c029cc8eb32585696120c6a2bf1f3625c8289b21
• GeoNetwork-side: Should be ready, tested locally with a issue751-branch mapfishapp POST'ing onto local GeoNetwork -> [OK]. It needed some extra work that I'd like to be validated by @fxprunayre though (so not proposing the work onto regular geonetwork for now)
  • The re-used stylesheet was generating a XML metadata which was not passing the schematron rules against the iso19139 schema, I rewrite some parts of it (adding the gmd: prefix everywhere, adding missing blocks, ...) but it could be done a better way (xsl templates ...).
  • <gmd:source> xlink:href should probably be excluded by the xlink:href resolution (or else we end up with OpenStreetMap wiki bundled into the MD using the sample in my unit test).

One thing that has not been done: Reprojecting the bounding box into 4326. This could be done in XslUtil.java.

But as for now, I think it is ready for a PR.

[fvanderbiest]

Reprojecting the bounding box into 4326

My reading of gmd/extent.xsd makes me think that it is required.

[pmauduit]

Done.

[fvanderbiest]

PR time ;-)

[fvanderbiest]

Deployed to sdi.geor.org, but the server responds with a 403

Remaining work: open new MD in new window. Response from service is

<?xml version="1.0" encoding="UTF-8"?>
<uuid>cf69805a-58cc-4eda-892a-1714735a2cd5</uuid>

[fvanderbiest]

OK, works now.

But:

• the metadata is published at once. One might want it to be reviewed before.
• it is not editable, because, in GN, it belongs to group "All".

Please suggest.

[pmauduit]

Almost ready, here is an early preview of the feature:

feedback welcome

[fvanderbiest]

Deployed to http://sdi.georchestra.org/mapfishapp/

Looks good, but:

• the context title should probably be mandatory, (looking for feedback on this, because it means that saving a context will always require a title),
• the resulting metadata does not explicitly associate "OGC WEB Map Context" with the WMC URL, cf

.. in addition to the above remarks.

[fvanderbiest]

The MD are assigned to MD_ScopeCode = mapDigital but I cannot find "mapDigital" in the "Resource type" combo when doing a search in the catalog.

In addition, it seems that the URL which opens up in the browser (eg http://sdi.georchestra.org/geonetwork/apps/georchestra/?uuid=e86f7fe8-40a0-4fd1-b42f-10f8793c4768 ) is not immediately valid, which results in GeoNetwork saying "I do not find MD uuid = e86f7fe8-40a0-4fd1-b42f-10f8793c4768"

[fvanderbiest]

@bchartier : we're looking for your feedback on the above questions. Thanks.

[fvanderbiest]

... but you'll have to ask an account on the demo server first : https://sdi.georchestra.org/ldapadmin/account/new

[pmauduit]

Done:

• mapDigital indexation
• OGC WMC label before the link (+ a link icon)

[fvanderbiest]

Links:

• https://github.com/georchestra/geonetwork/commit/b8d74bcdd4c450199148d434f804119745e55fda
• https://github.com/georchestra/geonetwork/commit/32ce5d002abd7a6d6212cbbc43443cf5d9a58ddf

Will require a GN data dir upgrade ... => release notes.

[fvanderbiest]

@pmauduit: could you update https://github.com/georchestra/geonetwork_minimal_datadir accordingly ? Thanks

[fvanderbiest]

the context title should probably be mandatory

I'll add a warning when the user saves the context to catalog if no title has been filled.

it seems that the URL which opens up in the browser is not immediately valid

Pierre suggests me to introduce a small delay before opening the popup.

[pmauduit]

@pmauduit: could you update https://github.com/georchestra/geonetwork_minimal_datadir accordingly ?

https://github.com/georchestra/geonetwork_minimal_datadir/commit/f6740b309a65ba3cfcddac7f5cb9da8fa9248f34

[fvanderbiest]

Looks good now, closing !

Thanks to all contributors.

[fvanderbiest]

Reopening the issue because it seems that GN only accepts members of SV_ADMIN to call the wmc.import service.

Expected: members of SV_EDITOR or SV_REVIEWER or SV_ADMIN have such right.

[pmauduit]

There might be some specific right management on the geonetwork side then, because:

https://github.com/georchestra/geonetwork/blob/georchestra-14.12/web/src/main/webapp/WEB-INF/config-security-mapping.xml#L222

Only the SV_ADMIN members might have the right to publish a md in the targeted group or so.

Since I'm porting the service to the develop branch of geonetwork trunk, I can have a look, but later on.

[pmauduit]

2 solutions for the problem "only the admin can publish onto the current targeted group":

• Create a new group, where every editors could create their MD into ;

or

• Get the group from the users (http://[IDS base url]/geonetwork/srv/eng/xml.info?type=groups) and propose the user to provide it when POST'ing to geonetwork.

I do prefer option 2.

[bchartier]

option 2 : +1 if the user could see the list of his/her groups and if he/she would have just to select one of them.

[pmauduit]

Done geonetwork-side: https://github.com/georchestra/geonetwork/commit/02303fbf9919914b11988ac2e36af7d7034e77c9

Waiting for @fvanderbiest mapfishapp-side