search

INSPIRE-MIF / helpdesk-validator

Community discussion forum for INSPIRE validation issues
42 stars 23 forks source link

ISO 19115 codelist locations #479

Closed PeterParslow closed 3 years ago

PeterParslow commented 3 years ago

Sadly, ISO don't seem to understand "persistence" in their authoritative XML schema or code lists locations. ISO TC211 have therefore established a separate authoritative set.

That is, https://standards.iso.org/iso/19139/resources/codelist/ returns an HTTP 404 Page not found. The actual file is at https://schemas.isotc211.org/schemas/19139/resources/gmxCodelists.xml

Meanwhile, the INSPIRE Metadata TG sneaked "shall" requirements into footnotes which seem to contradict (over ride) a general TG requirement, and the validator enforces these. See Footnote 21 referenced from TG Requirement C.6, 22 from C.10, 23 from C.13, 24 from C.15, 30 from C.21, 32 from TG Requirement 1.1

  1. Is the validator right to do that?

Against: INSPIRE Metadata TG Clause 2.1.1 third paragraph states

"Both the value of the codeList attribute (a URL that references a code list definition within a register or a code list catalogue) and the textual content of the ISO 19139 element are purely informative"

This is just below TG Requirement C.3 metadata/2.0/req/common/code-list-value, which states that it is the "codeListValue" attribute that shall be used to encode the code list value.

i.e. in Example 2.1

Creazione

it is "creation" that satisfies TG Requirement C.3

For: a number of footnotes with "shall"

"Attribute codeList shall be "http://standards.iso.org/iso/19139/resources/gmxCodelists.xml#CI_RoleCode"

Footnotes 25-29, about C.17 Limitations on public access & C.18 Conditions affecting access and use, don't say "shall"

Note: Footnote 19 referenced from TG Requirement C.5 (& 35 from 1.7) does the same for language code, but at least that URL still works (having an external ISO Maintenance Agency looking after it).

  1. Should the TG be changed to reflect the changed location of the authoritative code lists?
PeterParslow commented 3 years ago

Maybe I'm not quite right, because https://standards.iso.org/iso/19139/resources/gmxCodelists.xml#MD_ScopeCode works.

Is it just Example 2.1 & the paragraph above it that are wrong (by including "codelist" in the path)?

And of course the use of footnotes to over ride an earlier TG Requirement!

PeterParslow commented 3 years ago

However, even with exactly what the TG asks for:

                  <gmd:dateType>
                    <gmd:CI_DateTypeCode codeList="https://standards.iso.org/iso/19139/resources/gmxCodelists.xml#CI_DateTypeCode" codeListValue="publication">publication</gmd:CI_DateTypeCode>
                  </gmd:dateType>

my records fail validation, stating

"The metadata record contains a reference to a specification, but no dateType of 'creation', 'revision', or 'publication' is provided for it."

https://inspire.ec.europa.eu/validator/v2/TestRuns/EIDcd6463c1-2105-428f-8150-65b907be15e4.html

PeterParslow commented 3 years ago

This recurred several times this morning, and then for this record - with the same citation block copied & pasted - it passes: https://inspire.ec.europa.eu/validator/v2/TestRuns/EID53dae146-e7ef-4888-bcf9-b71c19aec91c.html

So perhaps the issue is something quite different?

carlospzurita commented 3 years ago

Dear @PeterParslow

Sorry for the delay in the response. We are unable to access the Test Reports you have linked in the issue, because they have been already deleted from the validator. Can you please provide us with the metadata you are using to test? That way we can replicate the issue ourselves.

Kind regards.

PeterParslow commented 3 years ago

As I mentioned in my last post, the same fragment in a different file passed. I'll have to run the test on the batch of them again to see what happens now.

Meanwhile, they have moved from our internal store to our WAF (which gets harvested to the national Discovery Service).

https://os.uk/xml/products/AddressBase.xml is an example

nmtoken commented 3 years ago

same issue as https://github.com/INSPIRE-MIF/helpdesk-validator/issues/69 ?

fabiovin commented 3 years ago

Dear @PeterParslow, I tested your metadata and the error on Req. C.21 is due to the fact that the citation of the "AddressBase Technical Specification" does not contain a "publication" date. Your metadata contains only a "revision" date, so you need to add also a "publication" date.

Regarding the "shall" requirements in footnotes about the codelists, we removed these checks in the validator and now the only check is on the "codeListValue" attribute.

So, your metadata will pass the validation also if you add the following:

                                            <gmd:dateType>
                                                <gmd:CI_DateTypeCode
                                                    codeList="anyURL"
                                                    codeListValue="publication"
                                                    >publication</gmd:CI_DateTypeCode>
                                            </gmd:dateType>

The error message is not clear and it does not help to understand the error. @carlospzurita Could you please improve the message by removing the two other date types?

It should be: "The metadata record contains a reference to a specification, but no dateType 'publication' is provided for it."

Thanks, Fabio

PeterParslow commented 3 years ago

I can do this, and I guess it would improve the record slightly (so long as I still also specify the revision date, which is more relevant to users - which version of the spec does it comply to?).

But surely C.21 is specific to citing the INSPIRE IR?

"Each gmd:report/gmd:DQ_DomainConsistency/gmd:result/gmd:DQ_ConformanceResult element shall include a citation of the INSPIRE Implementing Rule, specification document or Conformance Class, including its official title and the date of publication of the document, using gmd:specification/gmd:CI_Citation element."

I can't see that the same requirement applies to additional instances of DQ_ConformanceResult

fabiovin commented 3 years ago

Looking at the IR [Regulation 1205/2008], Part B 7:

7.1. Specification This is a citation of the implementing rules adopted under Article 7(1) of Directive 2007/2/EC or other specification to which a particular resource conforms.

it seems that C.21 applies to all the conformity declarations.

Probably we could open a dedicated discussion about that.

PeterParslow commented 3 years ago

To continue the quotation from https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2008:326:0012:0030:EN:PDF

7.1. Specification This is a citation of the implementing rules adopted under Article 7(1) of Directive 2007/2/EC or other specification to which a particular resource conforms. A resource may conform to more than one implementing rules adopted under Article 7(1) of Directive 2007/2/EC or other specification. This citation shall include at least the title and a reference date (date of publication, date of last revision or of creation) of the implementing rules adopted under Article 7(1) of Directive 2007/2/EC or of the specification.

So TG Rule C.21 is "tighter" than required by the regulation (which is OK) - the regulation could be satisfied by giving any of three dates of the IR. The Regulation doesn't actually require any date for citations of other specifications, and neither does the TG.

nmtoken commented 3 years ago

On another tack I disagree that all conformity declarations that can appear in metadata, are declarations of conformity to a specification, they can be used for a far broader set of declarations.

PeterParslow commented 3 years ago

James (@nmtoken) has a point. In ISO 19115, a "conformance result" is "Information about the outcome of evaluating the obtained value (or set of values) against a specified acceptable conformance quality level". By using it in the context of metadata scoped to a dataset (or series) it is a statement about the quality of that "set of values".

But in another sense, I'm not sure it is a point; in ISO 19115, the "a specified acceptable conformance quality level" that you cite conformance to is specifically called a "specification".

fabiovin commented 3 years ago

Dear all,

thank you for your comments.

We discussed internally and we decided to relax the test and to consider that the requirement about the conformity declaration (Regulation 1205/2008, Part B 7) applies only to "INSPIRE specifications" and not to all other possible specifications.

In addition, considering the following sentence in the IR (COMMISSION REGULATION (EC) No 1205/2008) and waiting for the implementation of a more quick revision process for the TGs, we decided to accept all types of dates, even if in the TG Requirement C.21 only the “publication” date is required.

This citation shall include at least the title and a reference date (date of publication, date of last revision or of
creation) of the implementing rules adopted under Article 7(1) of Directive 2007/2/EC or of the specification.
iuriemaxim commented 3 years ago

Dear all,

Wasn't this already covered here: https://github.com/INSPIRE-MIF/helpdesk-validator/issues/472

PeterParslow commented 3 years ago

@iuriemaxim - it looks similar, but different. This one is about the fact that the need for "publication date" should only apply when citing the INSPIRE spec. Conformance to other specs should be free to cite them in whatever way they choose.

472 is about the error message, but also the fact that people could choose to give a revision date in addition to the publication date, even for the INSPIRE spec.

The solution - relax the check - may be the same for both, but it need not be. It's OK to fix two (or more) issues with one fix.

arantzaetxebarria commented 3 years ago

Dear @PeterParslow

The ETS has been updated with a fix for this issue on the staging instance. Could you please test again and verify? Thanks

PeterParslow commented 3 years ago

Dear @arantzaetxebarria I can confirm that my record now passes validation on the staging instance.

And, by the way, I like that new interface.

Thanks

dperezBM commented 3 years ago

Dear @PeterParslow,

Thank you for your feedback. We will mark the issue as solved, but we are going to keep it open in order to monitor the issues that are attached to the next release. Please feel free to open a new issue if a new problem is raised.

Regards.