search

iris-edu / stationxml-validator

GNU General Public License v3.0
16 stars 8 forks source link

Snapshot 1.6.0.1 and current source only print help text #80

Closed ghost closed 4 years ago

ghost commented 4 years ago

I am testing against the attached file, current as of 2019-11-06: https://files.anss-sis.scsn.org/production/FDSNstationXML/IW/IW_MOOW.xml

If I execute version 1.5.9.5 it works and generates a report. java -jar station-xml-validator-1.5.9.5.jar IW_MOOW.2019110515.xml If I execute either the snapshot 1.6.0.1 or compile from source, it only prints the help message.

~/Downloads> java -jar stationxml-validator-1.6.0.1-SNAPSHOT.jar IW_MOOW.2019110515.xml                                                  (base) 
===============================================================
|                  FDSN StationXml validator                   |
|                  Version ${project.version}                  |
================================================================
Usage:
java -jar stationxml-validator <FILE> [OPTIONS]
OPTIONS
   --output         : where to output result, default is System.out
   --ignore-warnings: don't show warnings
   --rules          : print a list of validation rules
   --units          : print a list of units used to validate
   --debug          :
   --help           : print this message
===============================================================
 ~/Downloads>
timronan commented 4 years ago

Please add --format report to the end of your command. The complete command would look like

java -jar station-xml-validator-1.5.9.5.jar IW_MOOW.2019110515.xml --format report

We will work on updating the validator's internal and github documentation to reflect this. We may also remove the required format argument.

Thanks for the user experience feedback.

ghost commented 4 years ago

Adding --format report gives this:

jholland@igskgacgwssw000 ~/d/stationxml-validator> java -jar validator.jar /APPS/metadata/xml/IU.RSSD.xml --format report                                         (base) 
javax.xml.bind.UnmarshalException
 - with linked exception:
[org.xml.sax.SAXParseException; lineNumber: 68610; columnNumber: 32; cvc-datatype-valid.1.2.1: 'None' is not a valid value for 'double'.]
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.createUnmarshalException(AbstractUnmarshallerImpl.java:335)
    at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.createUnmarshalException(UnmarshallerImpl.java:578)
    at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:264)
    at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:229)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:157)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:204)
    at edu.iris.dmc.DocumentMarshaller.unmarshal(DocumentMarshaller.java:26)
    at edu.iris.dmc.Application.read(Application.java:150)
    at edu.iris.dmc.Application.run(Application.java:130)
    at edu.iris.dmc.Application.run(Application.java:121)
    at edu.iris.dmc.Application.main(Application.java:92)
Caused by: org.xml.sax.SAXParseException; lineNumber: 68610; columnNumber: 32; cvc-datatype-valid.1.2.1: 'None' is not a valid value for 'double'.
    at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:203)
    at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:134)
    at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)
    at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
    at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)
    at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator$XSIErrorReporter.reportError(XMLSchemaValidator.java:453)
    at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.reportSchemaError(XMLSchemaValidator.java:3231)
    at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.elementLocallyValidType(XMLSchemaValidator.java:3146)
    at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.processElementContent(XMLSchemaValidator.java:3056)
    at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleEndElement(XMLSchemaValidator.java:2135)
    at com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.endElement(XMLSchemaValidator.java:854)
    at com.sun.org.apache.xerces.internal.jaxp.validation.ValidatorHandlerImpl.endElement(ValidatorHandlerImpl.java:582)
    at com.sun.xml.bind.v2.runtime.unmarshaller.ValidatingUnmarshaller.endElement(ValidatingUnmarshaller.java:106)
    at com.sun.xml.bind.v2.runtime.unmarshaller.SAXConnector.endElement(SAXConnector.java:180)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:609)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2967)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:602)
    at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
    at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:505)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:842)
    at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:771)
    at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
    at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1213)
    at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:643)
    at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:258)
    ... 8 more
ghost commented 4 years ago

Further clarification. This did fix it for released version 1.6.0.1, but it gives the exception for the compiled version

ghost commented 4 years ago

This exception occurs when trying the latest compiled on a Dataless file.

jholland@igskgacgwssw000 ~/d/stationxml-validator> java -jar validator.jar  ~/DATALESS.IU_RSSD.seed --format report                                               (base) 
Exception in thread "main" java.lang.NoSuchMethodError: edu.iris.dmc.seed.control.station.Number.getValue()Ljava/lang/Double;
    at edu.iris.dmc.station.mapper.PolesZerosMapper.createFloatNoUnitType(PolesZerosMapper.java:215)
    at edu.iris.dmc.station.mapper.PolesZerosMapper.map(PolesZerosMapper.java:122)
    at edu.iris.dmc.station.converter.SeedToXmlDocumentConverter.convert(SeedToXmlDocumentConverter.java:232)
    at edu.iris.dmc.station.converter.SeedToXmlDocumentConverter.convert(SeedToXmlDocumentConverter.java:69)
    at edu.iris.dmc.Application.read(Application.java:153)
    at edu.iris.dmc.Application.run(Application.java:130)
    at edu.iris.dmc.Application.run(Application.java:121)
    at edu.iris.dmc.Application.main(Application.java:92)
ghost commented 4 years ago

Related_Files.zip

Summary of my testing: IRIS FDSN based IW_MOOW xml parses fine now. IRIS FDSN based IU_RSSD xml fails crashes when read. ASL created RSSD seed file crashes when read. (IRIS FDSN based RSSD xml converted to SEED also crashes when read same error)

timronan commented 4 years ago

The compiled version of the validator in the release directory has been updated. This deployed version/source code generated validator instances seem to be working correctly for the files provided in the related_files.zip directory. The FromIRIS_FDSN_IU.RSSD.xml file is generating the error described above because line 68610 has an InstrumentSensitivity:Value defined as the string "None" rather than a double (Refer to thw code block below). This fails the xml to xsd check which is embedded in the validator.

          <InstrumentSensitivity>
             <Value>None</Value>
             <Frequency>None</Frequency>
             <InputUnits>
               <Name>V</Name>
               <Description>Volts</Description>
             </InputUnits>
             <OutputUnits>
               <Name>COUNTS</Name>
               <Description>Digital Counts</Description>
             </OutputUnits>
           </InstrumentSensitivity>

How was the FromIRIS_FDSN_IU.RSSD.xml file generated/found? This file is not returned from IRIS' station services, and is not the file output from the stationxml-seed-converter if ASL_CREATED_DATALESS.IU_RSSD.seed is used as the input.

ghost commented 4 years ago

It was pulled from IRIS' fdsn interface via obspy. I can send the script we use to generate it if you need.

ghost commented 4 years ago

Thanks for looking into this. I appreciate it.

timronan commented 4 years ago

Can you please send the obspy script? Obspy may not be returning the metadata correctly and a bug may need to be filed. Try using https://service.iris.edu/fdsnws/station/1/query?net=IU&sta=RSSD&level=response&format=xml&includecomments=true&nodata=404 to get the IU RSSD metadata. The station services is the most direct way to access IRIS' data holdings.

ghost commented 4 years ago

I will get that to you momentarily.

Also something to note. Running

java -jar validator.jar /APPS/metadata/xml/IW.MOOW.xml --format html
java -jar validator.jar /APPS/metadata/xml/IW.MOOW.xml --RandomArgument

Or any other argument all result in the same behavior. The file being parsed as csv.

ghost commented 4 years ago

This should be the script generating the file.

Renamed to .txt to get around the .py being blocked. getdataless.txt

I agree about directly query the fdsn service. I will talk to the maintainer of the script about switching it.

timronan commented 4 years ago

I am still not finding where the FromIRIS_FDSN_IU.RSSD.xml file is created. The exception message associated with xml documents not passing xsd validation has been updated to: 

edu.iris.dmc.station.exceptions.StationxmlException: XML Document does not comply with the FDSN-StationXML xsd schema. 
Error occurs in the StationXML Document and is described by the line below (refer to trace for line #): 
cvc-maxExclusive-valid: Value '90.1' is not facet-valid with respect to maxExclusive '9.0E1' for type '#AnonType_LatitudeBaseType'.

    at edu.iris.dmc.DocumentMarshaller.unmarshal(DocumentMarshaller.java:33)
    at edu.iris.dmc.Application.read(Application.java:151)
    at edu.iris.dmc.Application.run(Application.java:130)
    at edu.iris.dmc.Application.run(Application.java:121)
    at edu.iris.dmc.Application.main(Application.java:92)
Caused by: org.xml.sax.SAXParseException; lineNumber: 24; columnNumber: 30; cvc-maxExclusive-valid: Value '90.1' is not facet-valid with respect to maxExclusive '9.0E1' for type '#AnonType_LatitudeBaseType'.
    at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(ErrorHandlerWrapper.java:204)
    at java.xml/com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.error(ErrorHandlerWrapper.java:135)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:396)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:327)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(XMLErrorReporter.java:284)
    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator$XSIErrorReporter.reportError(XMLSchemaValidator.java:511)
    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.reportSchemaError(XMLSchemaValidator.java:3587)
    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.elementLocallyValidComplexType(XMLSchemaValidator.java:3463)
    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.elementLocallyValidType(XMLSchemaValidator.java:3432)
    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.processElementContent(XMLSchemaValidator.java:3334)
    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.handleEndElement(XMLSchemaValidator.java:2372)
    at java.xml/com.sun.org.apache.xerces.internal.impl.xs.XMLSchemaValidator.endElement(XMLSchemaValidator.java:943)
    at java.xml/com.sun.org.apache.xerces.internal.jaxp.validation.ValidatorHandlerImpl.endElement(ValidatorHandlerImpl.java:584)
    at com.sun.xml.bind.v2.runtime.unmarshaller.ValidatingUnmarshaller.endElement(ValidatingUnmarshaller.java:106)
    at com.sun.xml.bind.v2.runtime.unmarshaller.SAXConnector.endElement(SAXConnector.java:180)
    at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:610)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1716)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2881)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.next(XMLNSDocumentScannerImpl.java:112)
    at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:534)
    at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:888)
    at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:824)
    at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
    at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1216)
    at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:635)
    at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal0(UnmarshallerImpl.java:258)
    at com.sun.xml.bind.v2.runtime.unmarshaller.UnmarshallerImpl.unmarshal(UnmarshallerImpl.java:229)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:137)
    at javax.xml.bind.helpers.AbstractUnmarshallerImpl.unmarshal(AbstractUnmarshallerImpl.java:184)
    at edu.iris.dmc.DocumentMarshaller.unmarshal(DocumentMarshaller.java:28)
    ... 4 more
ghost commented 4 years ago

I will try and track down the problem from our side. I am 90% sure that the script I provided generated the xml file, but it is possible it was running an old version of obspy or similar.

ghost commented 4 years ago

I am going to go ahead and close this ticket as I can get the validator to run now. I will open a more accurate ticket about how it ignores added options.