search

ome / bioformats

Bio-Formats is a Java library for reading and writing data in life sciences image file formats. It is developed by the Open Microscopy Environment. Bio-Formats is released under the GNU General Public License (GPL); commercial licenses are available from Glencoe Software.
https://www.openmicroscopy.org/bio-formats
GNU General Public License v2.0
373 stars 241 forks source link

Wrong timestamps extracted from CZI files (Z1 lightsheet) #2975

Open gatagat opened 6 years ago

gatagat commented 6 years ago

Hi,

bioformats fails to extract correct timestamps from CZI files produced by the Z1 light sheet. If i understand it correctly the timestamps (DeltaT) should report seconds elapsed since the beginning of the acquisition. As of now this is definitely not the case, see example below.

Issue #2946 regarding CZI files from Z1 is afaict not related.

Thanks for looking into this.

Best regards, Tomas

Here are tags produced by showinf -omexml -nopix (v5.7.1) for an input CZI file has 2 channels, 2 series, 399 time points, ~1 volume / minute. DeltaT should start at 0 and then keep increasing by ~60 which is not the case:

<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="0" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="1" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="2" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="3" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="4" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="5" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="6" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="7" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="8" TheZ="0"/>
<Plane DeltaT="2112.972056435075" DeltaTUnit="s" ... TheC="0" TheT="9" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="10" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="11" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="12" TheZ="0"/>
<Plane DeltaT="2112.972056435075" DeltaTUnit="s" ... TheC="0" TheT="13" TheZ="0"/>
<Plane DeltaT="2166.883157852748" DeltaTUnit="s" ... TheC="0" TheT="14" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="15" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="16" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="17" TheZ="0"/>
<Plane DeltaT="2112.972056435075" DeltaTUnit="s" ... TheC="0" TheT="18" TheZ="0"/>
<Plane DeltaT="2166.883157852748" DeltaTUnit="s" ... TheC="0" TheT="19" TheZ="0"/>
<Plane DeltaT="2220.9358700681382" DeltaTUnit="s" ... TheC="0" TheT="20" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="21" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="22" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="23" TheZ="0"/>
<Plane DeltaT="2112.972056435075" DeltaTUnit="s" ... TheC="0" TheT="24" TheZ="0"/>
<Plane DeltaT="2166.883157852748" DeltaTUnit="s" ... TheC="0" TheT="25" TheZ="0"/>
<Plane DeltaT="2220.9358700681382" DeltaTUnit="s" ... TheC="0" TheT="26" TheZ="0"/>
<Plane DeltaT="2274.956083516771" DeltaTUnit="s" ... TheC="0" TheT="27" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="28" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="29" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="30" TheZ="0"/>
<Plane DeltaT="2112.972056435075" DeltaTUnit="s" ... TheC="0" TheT="31" TheZ="0"/>
<Plane DeltaT="2166.883157852748" DeltaTUnit="s" ... TheC="0" TheT="32" TheZ="0"/>
<Plane DeltaT="2220.9358700681382" DeltaTUnit="s" ... TheC="0" TheT="33" TheZ="0"/>
<Plane DeltaT="2274.956083516771" DeltaTUnit="s" ... TheC="0" TheT="34" TheZ="0"/>
<Plane DeltaT="2328.6906762219155" DeltaTUnit="s" ... TheC="0" TheT="35" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="36" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="37" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="38" TheZ="0"/>
<Plane DeltaT="2112.972056435075" DeltaTUnit="s" ... TheC="0" TheT="39" TheZ="0"/>
<Plane DeltaT="2166.883157852748" DeltaTUnit="s" ... TheC="0" TheT="40" TheZ="0"/>
<Plane DeltaT="2220.9358700681382" DeltaTUnit="s" ... TheC="0" TheT="41" TheZ="0"/>
<Plane DeltaT="2274.956083516771" DeltaTUnit="s" ... TheC="0" TheT="42" TheZ="0"/>
<Plane DeltaT="2328.6906762219155" DeltaTUnit="s" ... TheC="0" TheT="43" TheZ="0"/>
<Plane DeltaT="2388.7523939093862" DeltaTUnit="s" ... TheC="0" TheT="44" TheZ="0"/>
<Plane DeltaT="1951.3366939368793" DeltaTUnit="s" ... TheC="0" TheT="45" TheZ="0"/>
<Plane DeltaT="2005.3592212845313" DeltaTUnit="s" ... TheC="0" TheT="46" TheZ="0"/>
<Plane DeltaT="2059.20608792738" DeltaTUnit="s" ... TheC="0" TheT="47" TheZ="0"/>
<Plane DeltaT="2112.972056435075" DeltaTUnit="s" ... TheC="0" TheT="48" TheZ="0"/>
<Plane DeltaT="2166.883157852748" DeltaTUnit="s" ... TheC="0" TheT="49" TheZ="0"/>

For simplicity showing only T<50, Z=0 C=0.)

dgault commented 6 years ago

Hi @gatagat, thank you for bringing this to our attention. If possible would you be able to upload the sample file for those reported values to https://www.openmicroscopy.org/qa2/qa/upload/?

gatagat commented 6 years ago

The file above is worth 1.5T, I'll try to reproduce the bug with a small test file and upload it.

gatagat commented 6 years ago

Hi @dgault, thanks for looking into this.

Here is the small test data (no sample) - ID 18093.

Expected: the extracted timestamps starting at 0, increasing monotonously up to about 34s for the four time points.

Actual:

<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="0" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="0" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="0" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="0" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="0" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="0" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="1" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="1" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="1" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="1" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="1" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="1" TheZ="2"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="0" TheT="2" TheZ="0"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="1" TheT="2" TheZ="0"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="0" TheT="2" TheZ="1"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="1" TheT="2" TheZ="1"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="0" TheT="2" TheZ="2"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="1" TheT="2" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="3" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="3" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="3" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="3" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="3" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="3" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="0" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="0" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="0" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="0" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="0" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="0" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="1" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="1" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="1" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="1" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="1" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="1" TheZ="2"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="0" TheT="2" TheZ="0"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="1" TheT="2" TheZ="0"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="0" TheT="2" TheZ="1"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="1" TheT="2" TheZ="1"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="0" TheT="2" TheZ="2"/>
<Plane DeltaT="276.2015836405138" DeltaTUnit="s" ... TheC="1" TheT="2" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="3" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="3" TheZ="0"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="3" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="3" TheZ="1"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="0" TheT="3" TheZ="2"/>
<Plane DeltaT="266.99418505689533" DeltaTUnit="s" ... TheC="1" TheT="3" TheZ="2"/>
dgault commented 6 years ago

Hi @gatagat, thank for you uploading the file. I have been able to test and debug it today and can confirm that Im able to reproduce the issue as described.

From debugging it seems that Bio-Formats has parsed 4 Timestamps attachments, 1 per file, from that it has parsed the following list of 10 timestamps :

266.99418505689533, 266.99418505689533, 276.2015836405138, 266.99418505689533, 276.2015836405138, 285.31430773753726, 266.99418505689533, 276.2015836405138, 285.31430773753726, 294.74838868852714

These 10 values were added from 1-4 values in each corresponding file: From test2.czi : 266.99418505689533 From test2(1).czi : 266.99418505689533, 276.2015836405138 From test2(2).czi : 266.99418505689533, 276.2015836405138, 285.31430773753726 From test2(3).czi : 266.99418505689533, 276.2015836405138, 285.31430773753726, 294.74838868852714

So the first issue we have is that it should be unique timepoints stored when the timepoints are split over multiple files like this.

The second issue is the actual values, if we assume the first timepoint is 0 and offset the remaining 3 values we would end up with: 0, 9.20739858362, 18.3201226806, 27.7542036316

This would align with what is displayed in Zen Black: 0, 9.21, 18.3, 27.8

So that at least explains the 2 issues, hopefully we can put a fix in place for these shortly

gatagat commented 6 years ago

Hi @dgault, is there anything new? Do you need any more information from our side?