search

ome / bioformats

Bio-Formats is a Java library for reading and writing data in life sciences image file formats. It is developed by the Open Microscopy Environment. Bio-Formats is released under the GNU General Public License (GPL); commercial licenses are available from Glencoe Software.
https://www.openmicroscopy.org/bio-formats
GNU General Public License v2.0
376 stars 242 forks source link

Problem with OME-XML validation #2705

Closed sebi06 closed 7 years ago

sebi06 commented 7 years ago

Hi guys,

I tried to validate an OME-TIFF using the bftools (5.3.1) but encountered the followwing problem.

1) Acquire CZI image and save to disk - OK 2) Open image an Fiji using BioFormats and save as OME-TIFF - OK 3) Create OME-XML using - OK

showinf -series 0 -no-upgrade -nometa -nopix -omexml-only -novalid -nocore Test.ome.tiff > Test.ome.xml

4) try to validate OME-XML --> but I could not get it to work, even with the no-upgrade option. What am I doing wrong here?

xmlvalid Test.ome.xml or xmlvalid -no-upgrade Test.ome.xml

First attempt:

Windows PowerShell Copyright (C) 2013 Microsoft Corporation. Alle Rechte vorbehalten.

PS C:\Users\M1SRH\Documents> cd .\Software\BioFormats_Package\5.3.1 PS C:\Users\M1SRH\Documents\Software\BioFormats_Package\5.3.1> xmlvalid c:\Output\Test.ome.xml Failed to compare version numbers java.net.SocketTimeoutException: connect timed out at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_111] at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) ~[na:1.8.0_111] at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) ~[na:1.8.0_111] at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) ~[na:1.8.0_111] at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_111] at java.net.PlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_111] at java.net.SocksSocketImpl.connect(Unknown Source) ~[na:1.8.0_111] at java.net.Socket.connect(Unknown Source) ~[na:1.8.0_111] at sun.net.NetworkClient.doConnect(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.New(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.New(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) ~[na:1.8.0_111] at loci.formats.UpgradeChecker.newVersionAvailable(UpgradeChecker.java:226) [bioformats_package.jar:na] at loci.formats.UpgradeChecker.newVersionAvailable(UpgradeChecker.java:171) [bioformats_package.jar:na] at loci.formats.tools.XMLValidate.main(XMLValidate.java:66) [bioformats_package.jar:na] Parsing schema path Error parsing schema path from c:\Output\Test.ome.xml org.xml.sax.SAXParseException: Content ist nicht zulõssig in Prolog. at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source) ~[na:1.8. 0_111] at com.sun.org.apache.xerces.internal.util.ErrorHandlerWrapper.fatalError(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.impl.XMLErrorReporter.reportError(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.impl.XMLScanner.reportFatalError(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl$PrologDriver.next(Unknown Source) ~[na:1.8.0_1 11] at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) ~[na:1.8. 0_111] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) ~[na:1.8.0_111] at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(Unknown Source) ~[na:1.8.0_111] at javax.xml.parsers.SAXParser.parse(Unknown Source) ~[na:1.8.0_111] at loci.common.xml.XMLTools.validateXML(XMLTools.java:623) ~[bioformats_package.jar:na] at loci.formats.tools.XMLValidate.process(XMLValidate.java:61) [bioformats_package.jar:na] at loci.formats.tools.XMLValidate.main(XMLValidate.java:89) [bioformats_package.jar:na] PS C:\Users\M1SRH\Documents\Software\BioFormats_Package\5.3.1>

And the second attempt:

PS C:\Users\M1SRH\Documents\Software\BioFormats_Package\5.3.1> xmlvalid -no-upgrade c:\Output\Test.ome.xml Failed to compare version numbers java.net.SocketTimeoutException: connect timed out at java.net.DualStackPlainSocketImpl.waitForConnect(Native Method) ~[na:1.8.0_111] at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source) ~[na:1.8.0_111] at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source) ~[na:1.8.0_111] at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source) ~[na:1.8.0_111] at java.net.AbstractPlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_111] at java.net.PlainSocketImpl.connect(Unknown Source) ~[na:1.8.0_111] at java.net.SocksSocketImpl.connect(Unknown Source) ~[na:1.8.0_111] at java.net.Socket.connect(Unknown Source) ~[na:1.8.0_111] at sun.net.NetworkClient.doConnect(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.openServer(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.New(Unknown Source) ~[na:1.8.0_111] at sun.net.www.http.HttpClient.New(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source) ~[na:1.8.0_111] at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source) ~[na:1.8.0_111] at loci.formats.UpgradeChecker.newVersionAvailable(UpgradeChecker.java:226) [bioformats_package.jar:na] at loci.formats.UpgradeChecker.newVersionAvailable(UpgradeChecker.java:171) [bioformats_package.jar:na] at loci.formats.tools.XMLValidate.main(XMLValidate.java:66) [bioformats_package.jar:na] Exception in thread "main" java.io.FileNotFoundException: -no-upgrade (Das System kann die angegebene Datei nicht finden ) at java.io.FileInputStream.open0(Native Method) at java.io.FileInputStream.open(Unknown Source) at java.io.FileInputStream.(Unknown Source) at java.io.FileInputStream.(Unknown Source) at loci.formats.tools.XMLValidate.main(XMLValidate.java:89) PS C:\Users\M1SRH\Documents\Software\BioFormats_Package\5.3.1>

sebi06 commented 7 years ago

I forgot to add the links for the test files:

https://dl.dropboxusercontent.com/u/623476/Test.ome.tiff

https://dl.dropboxusercontent.com/u/623476/Test.ome.xml

sbesson commented 7 years ago

Hi @sebi06, I think the issue with the ome.xml is related to its encoding

$ file -b --mime Test.ome.xml 
application/xml; charset=utf-16le
$ ~/bftools-5.3.1/xmlvalid Test.ome.xml 
Parsing schema path
Error parsing schema path from Test.ome.xml
org.xml.sax.SAXParseException: Content is not allowed in prolog.
    at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source) ~[bioformats_package.jar:5.3.1]
...

Changing the encoding back to UTF-8 restores the usage of xmlvalid:

$ iconv -f utf-16 -t utf-8 Test.ome.xml -o Test2.ome.xml
$ ~/bftools-5.3.1/xmlvalid Test2.ome.xml 
Parsing schema path
http://www.openmicroscopy.org/Schemas/OME/2015-01/ome.xsd
Validating Test2.ome.xml
...

Alternatively, the xmlvalid utility should work directly on the OME-TIFF and automatically retrieve the OME-XML block from the ImageDescription tag:

$ ~/bftools-5.3.1/xmlvalid Test.ome.tiff 
Parsing schema path
http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd
Validating Test.ome.tiff
No validation errors found.

The -no-upgrade option is not exposed in the xmlvalid utility. This is largely due to the fact that unlike showinf or bfconvert, the XML validation requires a connection to be able to access the schema. However adding this option should be possible if there is an use case. Let us know.

sebi06 commented 7 years ago

Hi,

so I order to export a valid OME-TIFF our ZEN Blue software (or anybody else) should write the OME-XML metadata using UTF-8 as an encoding? Did I get this one right?

And of course it would be nice at least for us to be able to use the xmlvalid tool without a connection to the internet. Especially for us we always have to use the -no-upgrade option because we are sitting behind a proxy.

sbesson commented 7 years ago

Hi @sebi06 and happy New Year,

you are absolutely right. As per the OME-TIFF specification, the encoding of the XML string stored in the ImageDescription tag should indeed be UTF-8.

The proxy use case makes sense and I recorded it in our existing card while reviewing offline validation. In the meantime, the showinf utility should validate the OME-XML by default when used in conjunction with the -omexml option so the following command

showinf -nopix -omexml -no-upgrade /path/to/file.ome.tiff

might provide a suitable workaround for your scenario.

sebi06 commented 7 years ago

Hi Sebastien,

I finally found some time to do some further testing and investigations. And we found some strange things where we need the help of you guys. All the info can be found here:

https://dl.dropboxusercontent.com/u/623476/OME-TIFF_Issues.zip

Here a short summary:

Remark: I used showinf on a windows 7 64bit system and someone told me that "piping" the output may cuse problems on widows ... (i have no idea here)

We checked our source code an we really write the XML encodes with UTF-8. Now the question is, what is going on here. Our OME-TIFF seems ok (see output of showinf), but when opened with the official BF 5.3.2 version in Fiji the output is corrupted.

Any help is really appreciated here.

Cheers, Sebi

dgault commented 7 years ago

Hi @sebi06 ,

Just an update on some progress with this. I can reproduce the issue you were seeing with the Fiji import when using a clean Fiji install. Through a bit of testing this seems to be caused when the bio-formats_plugins.jar is used with the individual component jars. Switching to the bioformats_package.jar (either the release jar or a built snapshot version) results in the correct display.

Debugging further the XML parsing error from the command line tools seems to relate to the µ character in this case. Though this is not part of the original XML and is not responsible for the Transform error you are seeing in Fiji. The transform error is seen when attempting to apply a transform to upgrade the XML from 2015 to 2016 specification. This transform process we are handling through the Xalan dependency. It seems likely that there may be some version mismatch occurring.

I will continue to investigate and keep you updated.

dgault commented 7 years ago

Some further testing on this. If I use the default Fiji install, but add the Xalan and Serializer 2.7.2 jar files to the Bio-Formats directory then the import runs without issue.

With these jars the transformer is loaded as expected: Loaded org.apache.xalan.transformer.TransformerImpl from file:/Applications/Fiji.app/jars/bio-formats/xalan-2.7.2.jar

Without the jars the transformer is loaded from the run time jar: Loaded com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl from /Applications/Fiji .app/java/macosx/jdk1.8.0_66/jre/Contents/Home/lib/rt.jar

From the rt.jar the versions used should be:

version.SAX=2.0
version.crimson=not-present
java.class.path=.
version.ant=not-present
version.DOM=3.0
version.xalan1=not-present
version.xalan2_2=Xalan Java 2.7.0
version.xerces2=Xerces-J 2.7.1
version.xerces1=not-present

I further tested downloading and using the Xalan 2.7.0 jar, could verify that the class was being loaded from the downloaded jar, but was unable to reproduce the issue this way.

sebi06 commented 7 years ago

Any news on this problem. I still cannot open the OME-TIFF exported from ZEN in Fiji correctly since the issues are still the same. Can I help somehow with more information?

dgault commented 7 years ago

Hi @sebi06,

We do not have any further updates on this problem at the moment. There is also a thread which I had opened on the ImageJ forum regarding this http://forum.imagej.net/t/bio-formats-dependency-issue/4148 but we will keep the discussion on this GitHub issue.

@ctrueden Do you perhaps have any thoughts on what may be causing the issue here?

ctrueden commented 7 years ago

@sbesson @dgault Sorry this slipped through the cracks for me.

Do you know if there have there been any recent changes to the Xerces or Xalan dependencies that might explain this?

Not so recent that I can remember them. However, it is true that ImageJ2 used to ship xalan and serializer, and it currently no longer does so. This problem likely began at the point when I removed those libraries from the distribution. I probably removed them because I did not understand why they were needed/useful, since that logic is included with the Java standard library, and nearly all Bio-Formats functionality works without them.

So I guess this is a bug in core Java, which Bio-Formats has worked around (for many years?) by depending directly on xalan and/or serializer? I would like to better understand the technical ramifications here, and rationale for requiring xalan and/or serializer, before plunging ahead and blindly readding those libraries to Fiji. Currently, the xalan and seralizer dependencies are excluded in pom-scijava to avoid infecting all downstream artifacts with those dependencies, which may not be necessary depending on your use case. These JARs are similar to SLF4J bindings and other pluggable implementations: here, they offer an alternative implementation of classes present in the core JRE, so they should be added only on the deployment side, rather than the Maven dependency side, to avoid inflicting that implementation decision on downstream consumers.

@melissalinkert Do you recall the initial circumstances surrounding the addition of these two libraries? I see that d306e34a2e87b334b4d3e897df6a6de5987df665 originally added these libraries, but there is no explanation in the history about why they were/are needed.

Has anyone tried looking for more general bug reports relating to this problem of XSLT transformations with and without these libraries on the classpath? It would be nice to have some confirmation from the greater Java community about whether this is a more general issue, or something weirdly specific to our code. Either way, it would be nice if we could work around the problem(s) somehow, avoiding the need for these extra dependencies.

sbesson commented 7 years ago

Hi @ctrueden, no worries and many thanks for your reply and additional hindsights.

As you rightfully mentioned, these libraries have been part of the Bio-Formats stack for almost the last decade. It is certainly possible some of the features used in the XSLT transforms in the OME model require this library in order to operate.

Reviewing the rationale behind the usage of the libraries and investigating whether they could be dropped in the mid-term in favor of the standard libraries makes complete sense. We will start this work and track progress in this public Trello card.

For downstream consumers like @sebi06, the immediate workaround is to use the uber-JAR (bioformats_package.jar) rather than the individual JARs - see http://forum.imagej.net/t/release-of-bio-formats-5-2-2/2760/7 for an example.

In the short term, depending on the policy decisions, either these dependencies could be re-included in the Fiji distribution - is this possible to do this without enabling them in pom-scijava? Alternatively we could consider shipping them via the Java8 update site alongside the other Bio-Formats Jars if that made sense.

Closing this issue as it is resolved from the Bio-Formats source code standpoint. However we can keep using the thread to finalize the decisions about the distribution.

sebi06 commented 7 years ago

Hi guys, so if I got you right this is technically not an BioFormats issue. And a workaround is to use the bioformats_package.jar within Fiji that can be downloaded directly from the BioFormats homepage, right?

This is fine with me and thnaks for digging taht deep into the issues since it is clear now that I can close this bug on our side since there is none in ZEN.

Sebi

ctrueden commented 7 years ago

so if I got you right this is technically not an BioFormats issue.

It is, and it isn't. It appears to be a bug in how core Java handles XSLT transformations. When a copy of the Apache Xalan library is included on the classpath, it gets automatically used for XSLT transformations instead of the built-in one, and then this problem does not happen.

we can keep using the thread to finalize the decisions about the distribution.

Like I said, I want to understand the problem more thoroughly before resorting to shipping Xalan with ImageJ2 or Fiji, since shipping Xalan has global ramifications for the entire distribution.

Thus, I started digging into the Xalan differences between the shipped implementation (com.sun.org.apache.xalan.internal, which purports to be version 2.7.0) and the official Apache one. I tested Bio-Formats on the CLI with xalan:xalan:2.7.0 and the XSLT transformation succeeds, whereas with the internal Java one, it fails.

The following minimal XML example fails with the internal Xalan implementation, but succeeds with Apache Xalan 2.7.0 or 2.7.2:

<?xml version="1.0" encoding="utf-8"?>
<OME xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2010-06 http://www.openmicroscopy.org/Schemas/OME/2010-06/ome.xsd" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.openmicroscopy.org/Schemas/OME/2010-06">
   <Instrument ID="Instrument:1">
      <LightSource ID="LightSource:1" />
   </Instrument>
</OME>

This XML was derived from @sebi06's example converted OME-TIFF. Note that the XML is invalid:

Parsing schema path
http://www.openmicroscopy.org/Schemas/OME/2010-06/ome.xsd
Validating ome-tiff.xml
cvc-complex-type.2.4.b: The content of element 'LightSource' is not complete. One of '{"http://www.openmicroscopy.org/Schemas/OME/2010-06":Laser, "http://www.openmicroscopy.org/Schemas/OME/2010-06":Filament, "http://www.openmicroscopy.org/Schemas/OME/2010-06":Arc, "http://www.openmicroscopy.org/Schemas/OME/2010-06":LightEmittingDiode}' is expected.
cvc-complex-type.2.4.b: The content of element 'LightSource' is not complete. One of '{"http://www.openmicroscopy.org/Schemas/OME/2010-06":Laser, "http://www.openmicroscopy.org/Schemas/OME/2010-06":Filament, "http://www.openmicroscopy.org/Schemas/OME/2010-06":Arc, "http://www.openmicroscopy.org/Schemas/OME/2010-06":LightEmittingDiode}' is expected.
Error validating document: 2 errors found

We can fix it as follows:

<?xml version="1.0" encoding="utf-8"?>
<OME xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2010-06 http://www.openmicroscopy.org/Schemas/OME/2010-06/ome.xsd" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.openmicroscopy.org/Schemas/OME/2010-06">
   <Instrument ID="Instrument:1">
      <LightSource ID="LightSource:1">
        <Laser />
      </LightSource>
   </Instrument>
</OME>

Unfortunately, it still fails to transform:

Attempting to update XML with version: 2010-06
XML updated to at least 2008-09
XML updated to at least 2009-09
XML updated to at least 2010-04
XML updated to at least 2010-06
Running UPDATE_201006 stylesheet.
XML updated to at least 2011-06
Running UPDATE_201106 stylesheet.

javax.xml.transform.TransformerConfigurationException: line 382: Attribute 'Visible' outside of element.
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.passWarningsToListener(TransformerFactoryImpl.java:818) [na:1.8.0_112]
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:996) [na:1.8.0_112]
    at loci.common.xml.XMLTools.getStylesheet(XMLTools.java:511) [ome-common-5.3.1.jar:5.3.1]
    at loci.formats.services.OMEXMLServiceImpl.transformToLatestVersion(OMEXMLServiceImpl.java:291) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.services.OMEXMLServiceImpl.createOMEXMLMetadata(OMEXMLServiceImpl.java:370) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.services.OMEXMLServiceImpl.createOMEXMLMetadata(OMEXMLServiceImpl.java:357) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.in.OMEXMLReader.initFile(OMEXMLReader.java:263) [formats-bsd-5.3.4.jar:5.3.4]
    at loci.formats.FormatReader.setId(FormatReader.java:1399) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.ImageReader.setId(ImageReader.java:839) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:651) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1024) [bio-formats-tools-5.3.4.jar:5.3.4]
    at loci.formats.tools.ImageInfo.main(ImageInfo.java:1110) [bio-formats-tools-5.3.4.jar:5.3.4]

javax.xml.transform.TransformerConfigurationException: line 383: Attribute 'Text' outside of element.
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.passWarningsToListener(TransformerFactoryImpl.java:818) [na:1.8.0_112]
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerFactoryImpl.newTemplates(TransformerFactoryImpl.java:996) [na:1.8.0_112]
    at loci.common.xml.XMLTools.getStylesheet(XMLTools.java:511) [ome-common-5.3.1.jar:5.3.1]
    at loci.formats.services.OMEXMLServiceImpl.transformToLatestVersion(OMEXMLServiceImpl.java:291) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.services.OMEXMLServiceImpl.createOMEXMLMetadata(OMEXMLServiceImpl.java:370) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.services.OMEXMLServiceImpl.createOMEXMLMetadata(OMEXMLServiceImpl.java:357) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.in.OMEXMLReader.initFile(OMEXMLReader.java:263) [formats-bsd-5.3.4.jar:5.3.4]
    at loci.formats.FormatReader.setId(FormatReader.java:1399) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.ImageReader.setId(ImageReader.java:839) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:651) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1024) [bio-formats-tools-5.3.4.jar:5.3.4]
    at loci.formats.tools.ImageInfo.main(ImageInfo.java:1110) [bio-formats-tools-5.3.4.jar:5.3.4]
XML updated to at least 2012-06
Running UPDATE_201206 stylesheet.
XML updated to at least 2013-06
Running UPDATE_201306 stylesheet.
XML updated to at least 2015-01
Running UPDATE_201501 stylesheet.

javax.xml.transform.TransformerException: An attribute whose value must be a QName had the value ''
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.postErrorToListener(TransformerImpl.java:804) [na:1.8.0_112]
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:745) [na:1.8.0_112]
    at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(TransformerImpl.java:351) [na:1.8.0_112]
    at loci.common.xml.XMLTools.transformXML(XMLTools.java:597) [ome-common-5.3.1.jar:5.3.1]
    at loci.common.xml.XMLTools.transformXML(XMLTools.java:577) [ome-common-5.3.1.jar:5.3.1]
    at loci.formats.services.OMEXMLServiceImpl.transformToLatestVersion(OMEXMLServiceImpl.java:329) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.services.OMEXMLServiceImpl.createOMEXMLMetadata(OMEXMLServiceImpl.java:370) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.services.OMEXMLServiceImpl.createOMEXMLMetadata(OMEXMLServiceImpl.java:357) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.in.OMEXMLReader.initFile(OMEXMLReader.java:263) [formats-bsd-5.3.4.jar:5.3.4]
    at loci.formats.FormatReader.setId(FormatReader.java:1399) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.ImageReader.setId(ImageReader.java:839) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.ReaderWrapper.setId(ReaderWrapper.java:651) [formats-api-5.3.4.jar:5.3.4]
    at loci.formats.tools.ImageInfo.testRead(ImageInfo.java:1024) [bio-formats-tools-5.3.4.jar:5.3.4]
    at loci.formats.tools.ImageInfo.main(ImageInfo.java:1110) [bio-formats-tools-5.3.4.jar:5.3.4]
Could not transform version 2010-06 OME-XML.

So, let's break down these transforms in more detail. If we first save the above minimal valid XML block as valid-2010-06.xml, then we can generate the others:

previous=""
for v in 2010-06 2011-06 2012-06 2013-06 2015-01 2016-06
do
test "$previous" &&
xsltproc ~/code/ome/ome-model/specification/src/main/resources/transforms/$previous-to-$v.xsl valid-$previous.xml > valid-$v.xml &&
diff valid-$previous.xml valid-$v.xml
previous="$v"
done

And now, as expected, older versions fail:

$ showinf valid-2013-06.xml
Checking file format [OME-XML]
Initializing reader
OMEXMLReader initializing valid-2013-06.xml
Populating metadata
Could not transform version 2013-06 OME-XML.
Failure during the reader initialization

But starting with 2015-01, it works:

jrun showinf valid-2015-01.xml
Checking file format [OME-XML]
Initializing reader
OMEXMLReader initializing valid-2015-01.xml
Populating metadata
Exception in thread "main" java.lang.IllegalArgumentException: Invalid series: 0

(Of course, there is a failure afterward since there are no <Image> tags, but it does get past XSLT conversion.)

So the question is: what is it specifically about 2013-06-to-2015-01.xsl that is problematic here? Before I dig further, it would be nice to know whether someone else has already done this investigation, and if so, what the findings are.

imagesc-bot commented 3 years ago

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/multichannel-ome-tiff-import-order-xyzct-vs-xyczt/45960/5