Open ywang-bom opened 9 years ago
Test Metadata for reproducing the bug
<?xml version="1.0" encoding="UTF-8"?>
<gmd:MD_Metadata xmlns:gmd="http://www.isotc211.org/2005/gmd"
xmlns:gco="http://www.isotc211.org/2005/gco"
xmlns:gml="http://www.opengis.net/gml/3.2"
xmlns:gmx="http://www.isotc211.org/2005/gmx"
xmlns:gts="http://www.isotc211.org/2005/gts"
xmlns:mdbase="http://www.bom.gov.au/wis/mdbase"
xmlns:geonet="http://www.fao.org/geonetwork"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.isotc211.org/2005/gmd http://www.ngdc.noaa.gov/metadata/published/xsd/schema/gmd/gmd.xsd http://www.isotc211.org/2005/gmx http://www.ngdc.noaa.gov/metadata/published/xsd/schema/gmx/gmx.xsd">
<gmd:fileIdentifier xmlns:gml="http://www.opengis.net/gml">
<gco:CharacterString>urn:x-wmo:md:au.gov.bom::ticket-32-test-1</gco:CharacterString>
</gmd:fileIdentifier>
<gmd:hierarchyLevel>
<gmd:MD_ScopeCode codeList="http://www.isotc211.org/2005/resources/codeList.xml#MD_ScopeCode"
codeListValue="series"/>
</gmd:hierarchyLevel>
<gmd:contact>
<gmd:CI_ResponsibleParty>
<gmd:organisationName>
<gco:CharacterString>BoM</gco:CharacterString>
</gmd:organisationName>
<gmd:role>
<gmd:CI_RoleCode codeList="http://www.isotc211.org/2005/resources/codeList.xml#CI_RoleCode"
codeListValue="pointOfContact"/>
</gmd:role>
</gmd:CI_ResponsibleParty>
</gmd:contact>
<gmd:dateStamp>
<gco:DateTime xmlns:gml="http://www.opengis.net/gml">2014-12-02T00:00:00</gco:DateTime>
</gmd:dateStamp>
<gmd:metadataStandardName>
<gco:CharacterString>WMO Core Metadata Profile of ISO 19115 (WMO Core), 2003/Cor.1:2006 (ISO 19115), 2007 (ISO/TS 19139)</gco:CharacterString>
</gmd:metadataStandardName>
<gmd:metadataStandardVersion>
<gco:CharacterString>1.3</gco:CharacterString>
</gmd:metadataStandardVersion>
<gmd:identificationInfo>
<gmd:MD_DataIdentification>
<gmd:citation>
<gmd:CI_Citation>
<gmd:title>
<gco:CharacterString>Ticket 32: test metadata 1</gco:CharacterString>
</gmd:title>
<gmd:date>
<gmd:CI_Date>
<gmd:date>
<gco:Date>1970-01-01</gco:Date>
</gmd:date>
<gmd:dateType>
<gmd:CI_DateTypeCode codeList="http://www.isotc211.org/2005/resources/codeList.xml#CI_DateTypeCode"
codeListValue="publication"/>
</gmd:dateType>
</gmd:CI_Date>
</gmd:date>
</gmd:CI_Citation>
</gmd:citation>
<gmd:abstract>
<gco:CharacterString>abstract</gco:CharacterString>
</gmd:abstract>
<gmd:descriptiveKeywords>
<gmd:MD_Keywords id="WMOCodeListKeywords">
<gmd:keyword>
<gco:CharacterString>meteorology</gco:CharacterString>
</gmd:keyword>
<gmd:type>
<gmd:MD_KeywordTypeCode codeList="http://www.isotc211.org/2005/resources/codeList.xml#MD_KeywordTypeCode"
codeListValue="theme"/>
</gmd:type>
</gmd:MD_Keywords>
</gmd:descriptiveKeywords>
<gmd:resourceConstraints>
<gmd:MD_LegalConstraints>
<gmd:useLimitation gco:nilReason="missing">
<gco:CharacterString />
</gmd:useLimitation>
<gmd:accessConstraints>
<gmd:MD_RestrictionCode codeList="http://www.isotc211.org/2005/resources/codeList.xml#MD_RestrictionCode"
codeListValue="otherRestrictions"/>
</gmd:accessConstraints>
<gmd:useConstraints>
<gmd:MD_RestrictionCode codeList="http://www.isotc211.org/2005/resources/codeList.xml#MD_RestrictionCode"
codeListValue="otherRestrictions"/>
</gmd:useConstraints>
<gmd:otherConstraints>
<gco:CharacterString>Non-commerical use only</gco:CharacterString>
</gmd:otherConstraints>
</gmd:MD_LegalConstraints>
</gmd:resourceConstraints>
<gmd:language>
<gmd:LanguageCode codeList="http://www.isotc211.org/2005/resources/codeList.xml#LanguageCode"
codeListValue="eng"/>
</gmd:language>
<gmd:topicCategory>
<gmd:MD_TopicCategoryCode>climatologyMeteorologyAtmosphere</gmd:MD_TopicCategoryCode>
</gmd:topicCategory>
<gmd:extent>
<gmd:EX_Extent id="boundingExtent">
<gmd:geographicElement>
<gmd:EX_GeographicBoundingBox>
<gmd:westBoundLongitude>
<gco:Decimal>-180.0</gco:Decimal>
</gmd:westBoundLongitude>
<gmd:eastBoundLongitude>
<gco:Decimal>180.0</gco:Decimal>
</gmd:eastBoundLongitude>
<gmd:southBoundLatitude>
<gco:Decimal>-90.0</gco:Decimal>
</gmd:southBoundLatitude>
<gmd:northBoundLatitude>
<gco:Decimal>90.0</gco:Decimal>
</gmd:northBoundLatitude>
</gmd:EX_GeographicBoundingBox>
</gmd:geographicElement>
</gmd:EX_Extent>
</gmd:extent>
</gmd:MD_DataIdentification>
</gmd:identificationInfo>
</gmd:MD_Metadata>
Updated to clarify the nature of the bug as requested during 2014-11-28 OpenWIS TMC
Thanks for the clarification: we reproduced the bug. After checking the issue, Remy and I think your possible fix might be a good solution.
Hi Francois,
I have added the links to the binaries here which also hosts the latest jenkins builds
https://repository-openwis-association.forge.cloudbees.com/openwis/
HI Dom
Please can you check the status of this issue recently revisited in the TC meeting. Please can you attach the metadata record that was used to produce the issue with non wmo data for further investigation by Francois
Thanks
I used the same sample that Yang provided, but here it is as an attachment. nonWMO.zip (I've had to add it as a .zip as .xml is not supported in this window)
Excellent, thanks Dom, I shall assign this back to Francois and let Remy know as well.
I am confused by the term nonWMO. If I understand correctly,this is a metadata which is not for GlobalExchange. OK. However, I believe that a metadata compliant with WCMP 1.3 must have a data policy attached WMOOther for example.As far as I know, this is a must. In the example sent by Dom, I can't see that. So, for me, it is not a WMO complient metadata hence, it is quite 'normal' that OpenWIS refuses to use that. Am I missing something? Or, what is a nonWMO metadata?
@remygiraud The metadata in question is for local/regional data, i.e. not to be exchanged globally and not to be Cached. The WMO core metadata profile 1.3 requires the data policy/license code (i.e. WMOOther etc.) to be present only for global exchanged products (9.3.1). Since the metadata is NOT for global exchange (it could be RegionalExchange
or OriginatingCentre
or N/A), it is not required to have an associated policy/license code and is still a valid WCMP 1.3 metadata.
the metadata urn:x-wmo:md:au.gov.bom::ticket-32-test-1 is rejected by OpenWIS portal configured as follows (default) in openwis-metadataportal.properties:
openwis.metadataportal.extract.xpath=gmd:identificationInfo/gmd:MD_DataIdentification/gmd:resourceConstraints/gmd:MD_LegalConstraints/gmd:otherConstraints/gco:CharacterString,\
gmd:identificationInfo/gmd:MD_DataIdentification/gmd:resourceConstraints/gmd:MD_LegalConstraints/gmd:useLimitation/gco:CharacterString
the metadata urn:x-wmo:md:au.gov.bom::ticket-32-test-1 enters the catalog of a portal (with openwis 3.14.x and other versions) configured as follows in openwis-metadataportal.properties:
openwis.metadataportal.extract.xpath=gmd:identificationInfo/gmd:MD_DataIdentification/gmd:resourceConstraints/gmd:MD_LegalConstraints/gmd:useLimitation/*,\
gmd:identificationInfo/gmd:MD_DataIdentification/gmd:resourceConstraints/gmd:MD_LegalConstraints/gmd:otherConstraints/*
You can see an example with this url: http://dcpc-vaac-int.meteo.fr/openwis-user-portal/srv/en/main.home?urn=urn:x-wmo:md:au.gov.bom::ticket-32-test-1 This configuration is a Meteo France choice (because of DCPCs, Inspire compliance, ...). I think that many others can work.
This issue was moved to OpenWIS/openwis4#40
reopened since OpenWIS v3.x is in maintenance mode for next 6 years so this still applies
NOTE: This bug concerns metadata records NOT describing data for global exchange, i.e. the data do NOT have a corresponding WMO Data License Code (e.g. WMOEssential).
OpenWIS refuses to create a metadata record because the system derives an empty string as the "GTS Category" from the given XML file. The system reports error "Unable to extract OpenWIS information" shown as follows:
Steps to reproduce
Analysis From our investigation, OpenWIS has a set of logic to derive the value of "GTS Category" for a given XML file. The relevant method is
extractGTSCategoryGTSPriorityAndDataPolicy
inProductMetadataISO19139Extractor.java
. (The "GTS Category" corresponds to the "gts_category" column of "openwis_product_metadata" table in database)The logic to derive "GTS Category" from a given file is list as follows:
//gmd:identificationInfo//gmd:resourceConstraints//gmd:useLimitation
*//gmd:identificationInfo//gmd:resourceConstraints//gmd:otherConstraints
Relevant config file is
openwis-metadataportal.properties
, which instructs OpenWIS to use the above two elements to derive "GTS Category".The error is generated when:
gmd:useLimitation
andgmd:otherConstraints
, can be found in the file.gmd:useLimitation
is an empty element (i.e.<gmd:useLimitation gco:nilReason="missing"> <gco:CharacterString /> </gmd:useLimitation>
)gmd:useLimitation
is the last element in the loop.The method
extractGTSCategoryGTSPriorityAndDataPolicy
will then set "GTS Category" to be an empty string, which is exactly the content ofgmd:useLimitation
. This in turn triggers the reported error generated byensureProductMetadataValid
inMetadataAligner.java
.Possible Fix A possible fix is to check the "GTS Category" value at the end of the function when the given XML is NOT for global exchanged data. If the value is an empty string, simply set it to "Unknown" similar to step 2 in above logic.