search

soft-matter / pims

Python Image Sequence: Load video and sequential images in many formats with a simple, consistent interface.
http://soft-matter.github.io/pims/
Other
258 stars 67 forks source link

Bioformats and .oir files #274

Open aaron-cognitiv opened 6 years ago

aaron-cognitiv commented 6 years ago

1) I'm interested in loading .oir files using your BioformatsReader. OME Bioformats added support for Olympus's .oir files with version 5.5. By upgrading the loci_tools.jar file that Pims uses, I was able to successfully read most .oir files. Therefore, I wonder if you can officially upgrade the bioformats version that pims downloads/uses?

2) When .oir files reach a size larger than 1Gb, Olympus automatically begins splitting the file into subfiles (i.e. 'datafile.oir' will contain time points 0 through 399, then datafile_00001 will contain time points 400 through 799, etc...). However, when I attempt to use the BioformatsReader to load a subfile, I receive an Exception:

347             self._metadata = loci.formats.MetadataTools.createOMEXMLMetadata()
    348             self.rdr.setMetadataStore(self._metadata)
--> 349         self.rdr.setId(self.filename)
    350         if meta:
    351             self.metadata = MetadataRetrieve(self._metadata)

java.lang.OutOfMemoryErrorPyRaisable: java.lang.OutOfMemoryError: Java heap space

This exception appears to relate to the use of the ChannelSeparator object to parse the subfile, since the python-bioformat's ImageReader object can successfully load the subfiles.

P.S. Pims is a great package. Thank you.

caspervdw commented 6 years ago

Thanks @aaron-cognitiv for using our package and reporting your issues! I am happy that the BioformatsReader works for your Olympus files. Concerning your points

  1. I have made a PR to update to 5.4.1 (#266) however this gives issues with different file types (nd2). I silently hope that this issue will go away in a future (or current?) version of Bioformats. I will check up on that.

  2. Memory issues can occur using the java interface, because it defines a default heap size of 512 MB. The easiest option is increasing the heap size using BioformatsReader(<filename>, java_memory='1024m') or more.

aaron-cognitiv commented 6 years ago

Thanks

  1. Perhaps 5.6 has solved the issue.

  2. Unfortunately, the memory exception is a red-herring. Increasing the heap size does not help, and I can successfully open much larger files with current heap size. The problem seems to be that the bioformats reader does work when opening .oir sub-files. I'm communicating on the bioformats forum in hopes of getting this resolved.

caspervdw commented 6 years ago
  1. Nope, also persisting on 5.7.0

Tagging https://github.com/openmicroscopy/bioformats/issues/2955