search

Arcadia-Science / readlif

Leica Image Format (LIF) file reader for Python
GNU General Public License v3.0
32 stars 13 forks source link

Associate image metadata with LifImage, XZ plane error #18

Closed nanthony21 closed 3 years ago

nanthony21 commented 3 years ago

Tried this package after having a hard time getting started with python-bioformats, but I didn't get very far.

It appears that item.find for getting dim_y is returning None

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.5\plugins\python-ce\helpers\pydev\pydevd.py", line 1477, in _exec
    pydev_imports.execfile(file, globals, locals)  # execute the script
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.3.5\plugins\python-ce\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "C:/Users/backman05/Documents/Bitbucket/Confocal_NN_SurfaceTracking/src/fileHandling/lif.py", line 12, in <module>
    f = LifFile(filePath)
  File "C:\Users\backman05\Anaconda3\envs\confocal\lib\site-packages\readlif\reader.py", line 552, in __init__
    self.image_list = self._recursive_image_find(self.xml_root)
  File "C:\Users\backman05\Anaconda3\envs\confocal\lib\site-packages\readlif\reader.py", line 353, in _recursive_image_find
    self._recursive_image_find(item, return_list, appended_path)
  File "C:\Users\backman05\Anaconda3\envs\confocal\lib\site-packages\readlif\reader.py", line 365, in _recursive_image_find
    dim_y = int(item.find(
AttributeError: 'NoneType' object has no attribute 'attrib'

On closer inspection this is occurring for an image that is an XZ scan. Adding the same try:except handling that is used for other dimensions seems to have fixed it.

nimne commented 3 years ago

Hello - you've found a specific deficiency in the library at the moment, although this is a good time to find that specific one! Thanks for opening this issue! This is the same issue as described over in #14.

While the try-except handing will remove the error, I'm not sure you'll always be able to get the data that you expect. If you're not afraid to try out some in-progress code, you can look through the discussion in #14 and check out the development branch. With any luck (and if I get some free time), that specific problem will probably be solved in a few weeks.

Be warned that if you do use the development branch, there are issues - it currently doesn't work with more than one channel, and the code / API in the dev branch likely to change at any moment. I suggest you grab this specific commit for something that works (very slowly and on one channel). Also be warned that I will sometimes get non-working code up on the dev branch by accident, and the documentation isn't up to date.

In the dev branch, there is a new function for accessing the data that is explained here. Note that the name of the function will almost certainly be changed to get_slice() in the next commit (hopefully sometime this weekend).

nanthony21 commented 3 years ago

Thanks for getting back to me and thanks for your work on what I hope will become a very useful piece of code. I'll take a look at #14

On an unrelated note, one area that would be nice to have improved is the ease of access to metadata. I'm currently trying to extract the translation stage XY coordinates for each image from LifFile.xml_header but it's turning out to be difficult to do properly.

nimne commented 3 years ago

I agree.. the XML metadata is a big mess. Right now, the only things that are extracted are what is relevant to the data that is returned. While I don't have an immediate solution, look in to element trees which is how the reader parse things during loading. The element tree representation is returned as xml_root.

You raise a valid point that access to image metadata is difficult. Do you think it would be helpful if each image (or plane) had it's own associated XML metadata? Right now the xml in LifFile is for everything which is daunting. It should be pretty simple to return the ElementTree 'root' for each image when the reader parses it? This would end up adding a xml_root for every LifImage. From that, you could probably find the metadata you're looking for.

nanthony21 commented 3 years ago

I'm not familiar enough with the specifics of XML to know the best way to go about this but in the Leica LAS AF software you can view the properties of each individual image and it shows a table of all the recorded values from the connected hardware.

If there was a way get all that same information, preferably as a dict from the LifImage class I think that would be really convenient.

nimne commented 3 years ago

Good to know! I'll play around with that after I get the XZ scan problem worked out.

It would help me to know, what specific information is there? Unfortunately I don't have quick access to Leica software / a scope at the moment!

nimne commented 3 years ago

Hi, sorry it's been such a long time since I've had a chance to look at the metadata request. The problem that I'm running into is the sheer amount of data that is associated with each image. The XML from a single image (not the whole file) is below.

Representing all of this data as a dict isn't straightforward, particularly due to the repetitive and structured nature of some of the data. For example, the two channels from the image shown below are represented by two ChannelDescription elements. For each image, it's pretty easy to return the ElementTree.Element for each image in something like a LifImage.metadata_ET, but running through that big XML representation still takes some knowledge about what you're looking for.

If you have any creativity to add to the solution, I would appreciate it! Otherwise, I'm a little stuck with how to implement this.

  <Element Name="xzt" Visibility="1" CopyOption="1" UniqueID="24d1e2f4-7d08-11eb-97e9-0015774389b1">
    <Data>
      <Image TextDescription="">
        <Attachment Name="PreviewMarker" Application="LAS AF" IsPreviewImage="0"/>
        <ImageDescription>
          <Channels>
            <ChannelDescription DataType="0" ChannelTag="0" Resolution="8" NameOfMeasuredQuantity="" Min="0.000000e+000" Max="2.550000e+002" Unit="" LUTName="Green" IsLUTInverted="0" BytesInc="0" BitInc="0"/>
            <ChannelDescription DataType="0" ChannelTag="0" Resolution="8" NameOfMeasuredQuantity="" Min="0.000000e+000" Max="2.550000e+002" Unit="" LUTName="Red" IsLUTInverted="0" BytesInc="16384" BitInc="0"/>
          </Channels>
          <Dimensions>
            <DimensionDescription DimID="1" NumberOfElements="128" Origin="1.376765e-020" Length="1.781554e-005" Unit="m" BitInc="0" BytesInc="1"/>
            <DimensionDescription DimID="3" NumberOfElements="128" Origin="4.413365e-006" Length="1.781540e-005" Unit="m" BitInc="0" BytesInc="128"/>
            <DimensionDescription DimID="4" NumberOfElements="20" Origin="0.000000e+000" Length="1.007000e+001" Unit="s" BitInc="0" BytesInc="32768"/>
          </Dimensions>
        </ImageDescription>
        <TimeStampList NumberOfTimeStamps="40">1d71114eb8b44f0 1d71114eb8b44f0 1d71114ebdc2410 1d71114ebdc2410 1d71114ec2d0330 1d71114ec2d0330 1d71114ec7de250 1d71114ec7de250 1d71114eccec170 1d71114eccec170 1d71114ed1fa090 1d71114ed1fa090 1d71114ed707fb0 1d71114ed707fb0 1d71114edc15ed0 1d71114edc15ed0 1d71114ee123df0 1d71114ee123df0 1d71114ee631d10 1d71114ee631d10 1d71114eeb3fc30 1d71114eeb3fc30 1d71114ef04db50 1d71114ef04db50 1d71114ef55ba70 1d71114ef55ba70 1d71114efa69990 1d71114efa69990 1d71114eff778b0 1d71114eff778b0 1d71114f04857d0 1d71114f04857d0 1d71114f09936f0 1d71114f09936f0 1d71114f0ea1610 1d71114f0ea1610 1d71114f13af530 1d71114f13af530 1d71114f18bd450 1d71114f18bd450 </TimeStampList>
        <Attachment Name="ChannelAttachment" Application="LAS AF"/>
        <Attachment Name="ViewerScaling" Application="LAS AF">
          <ChannelScalingInfo BackgroundLutName="" WhiteValue="0.999009900990099" BlackValue="0" GammaValue="1" Automatic="0"/>
          <ChannelScalingInfo BackgroundLutName="" WhiteValue="1" BlackValue="0" GammaValue="1" Automatic="0"/>
        </Attachment>
        <Attachment Name="ImageXMLDocument" Application="LAS AF">
          <RootNode/>
        </Attachment>
        <Attachment Name="HardwareSetting" Application="LAS AF" Software="LAS X 3.5.7.23225" HardwareServerVersion="Build 23225" SystemType="12" SystemTypeName="TCS SP8" DataSourceType="0" DataSourceTypeName="Confocal">
          <ATLConfocalSettingDefinition VersionNumber="15" UserSettingName="S11" CanDoSTED="0" IsSTEDActive="0" IsSwitchForPulsedStedWithMpActive="0" UseSystemOptimizedVoxelCalculation="1" IsUserSettingNameSet="0" IsHydPulsedModeCoeffntManualActive="0" XGalvoMovementMode="0" XGalvoMovementModeName="sinus" BitSize="8" MaxIntegrationTime="0" ScanMode="xzt" ZUseMode="1" ZUseModeName="z-galvo" ZPosition="1.4664631328599E-10" IsSuperZ="0" CycleCount="20" CycleTime="0.533" CompleteTime="10.657" LineTime="0.0025" FrameTime="0.53" UseMaxIterationsForT="0" IsTimeMinimizeEnabled="0" LastTCalcMode="3" StagePosX="0.03462093930532" StagePosY="0.01474305397499" StageRangeX="0.07152023681912" StageRangeY="0.02913209445936" FlipX="0" FlipY="1" SwapXY="1" Magnification="63" ObjectivePos="1" ObjectiveName="HC PL APO CS2    63x/1.30 GLYC " ObjectiveNumber="11506353" MicroscopeModel="DMI6000B-CS" IsInverseMicroscopeModel="1" Immersion="GLYC" NumericalAperture="1.3" RefractionIndex="1.46" IsFCSFilterAutomationActive="0" IsSMDChaserUVAOTFAutomationActive="0" CanDoCSMode="1" ActiveCS_SubModeForTLD="0" ActiveCS_SubModeForTLDName="Scan-BF" ActiveCS_SubModeForRLD="0" ActiveCS_SubModeForRLDName="Empty" RldFluoCubePos="3" RldCubeAutoSelectionEnabled="1" LastNonMP_MFP_FW_Name="" LastNonMP_Pol_FW_Name="" ScanSpeed="400" InDimension="128" OutDimension="128" UseScanFormatRestriction="0" Zoom="10.3574655270055" BaseZoom="0.75" PanFirstDim="8.67361737988404E-19" PanSecondDim="4.41336526552075E-06" ScanDirectionX="1" ScanDirectionXName="Unidirectional" RotatorAngle="0" Pinhole="8.27950653654944E-05" PinholeAiry="0.804818919719259" EmissionWavelengthForPinholeAiryCalculation="580" FrameAverage="1" LineAverage="1" FrameAccumulation="1" Line_Accumulation="1" AreBleachPointsEnabled="0" InTrigger="-1" OutTrigger="-1" IsRoiScanEnable="0" Is3DLimitedRoiScanEnable="0" ZCompensationModes="0" WizardMode="0" IsConstantIntegrationTimeActive="1" PixelDwellTime="0.000004875" SystemSerialNumber="8100000201">
            <EasySRMappingArray/>
            <EasySRSlider SRRelative="75" ThreeDRelative="0" GentleRelative="25"/>
            <Quantity Value="2.28984615384615E-07" Unit=""/>
            <AdditionalZPositionList>
              <AdditionalZPosition Valid="0" SuperZMode="1" SuperZModeName="RestrictedRange" ZMode="1" ZUseModeName="z-galvo" ZPosition="0"/>
              <AdditionalZPosition Valid="1" SuperZMode="1" SuperZModeName="RestrictedRange" ZMode="2" ZUseModeName="z-wide" ZPosition="0.0039685414"/>
            </AdditionalZPositionList>
            <ShutterList>
              <Shutter Version="0" LightSourceType="4" LightSourceName="SuperContVisible2" IsActive="1">
                <BeamRoute Version="0">
                  <BeamPosition BeamPositionLevel="0" BeamPosition="10"/>
                  <BeamPosition BeamPositionLevel="1" BeamPosition="1"/>
                </BeamRoute>
              </Shutter>
            </ShutterList>
            <BleachPoints>
              <LMSDataContainerHeader Version="2">
                <Element Name="BleachPointROISet" Visibility="2" CopyOption="1" UniqueID="2f63c0db-7d08-11eb-97e9-0015774389b1">
                  <Data>
                    <ROISet ROISetType="1" PossibleChildROITypes="-1" PossibleROITransforms="65535" PossibleROIActions="65535"/>
                  </Data>
                  <Memory Size="0" MemoryBlockID="MemBlock_79"/>
                  <Children/>
                </Element>
              </LMSDataContainerHeader>
            </BleachPoints>
            <ATLConfocalBleachPointsSettings/>
            <FilterWheel FluorifierAutoSelection="1" BeamSplitterAutoSelection="0" StedBeamSelectionPinholeCoupling="1" MultiFunctionPortAutoSelection="1" BeamMergerAutoSelectionEnabled="0" RldBlockingFilterAutoSelectionEnabled="0">
              <Wheel Version="0" Qualifier="140" FilterWheelName="External Detection FW" FilterIndex="2" FilterName="Mirror " IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="160" FilterWheelName="Galvo Slider" FilterIndex="0" FilterName="Galvo X Normal " IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="180" FilterWheelName="Multi Function Port" FilterIndex="4" FilterName="Substrate " IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0">
                  <BeamPosition BeamPositionLevel="0" BeamPosition="20"/>
                </BeamRoute>
              </Wheel>
              <Wheel Version="0" Qualifier="131" FilterWheelName="Notch FW 2" FilterIndex="0" FilterName="Empty" IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="130" FilterWheelName="Polarization FW" FilterIndex="0" FilterName="Empty" IsSpectrumTurnMode="0" FilterSpectrumPos="74096" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="200" FilterWheelName="Galvo Resonant Pan" FilterIndex="0" FilterName="Galvo X Pan Center " IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="219" FilterWheelName="STED Beam Selection" FilterIndex="0" FilterName="STED Beam 1 " IsSpectrumTurnMode="0" FilterSpectrumPos="2250" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="220" FilterWheelName="STED Cleanup Slider" FilterIndex="0" FilterName="Empty" IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="221" FilterWheelName="STED Beam Slider" FilterIndex="0" FilterName="Substrate " IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="216" FilterWheelName="STED Phase Filter Beam 1" FilterIndex="0" FilterName="Empty" IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="217" FilterWheelName="STED Phase Filter Beam 2" FilterIndex="0" FilterName="Empty" IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
              <Wheel Version="0" Qualifier="170" FilterWheelName="Target Slider" FilterIndex="0" FilterName="Target Park " IsSpectrumTurnMode="0" FilterSpectrumPos="0" IndexChanged="0" SpectrumChanged="0">
                <BeamRoute Version="0"/>
              </Wheel>
            </FilterWheel>
            <Spectro>
              <MultiBand Channel="1" ChannelName="Channel 1" LeftWorld="492.993555750951" RightWorld="541.093325251024" TargetWaveLengthBegin="492.993555750951" TargetWaveLengthEnd="541.093325251024" DyeName="Leica/FITC"/>
              <MultiBand Channel="2" ChannelName="Channel 2" LeftWorld="548.65512503415" RightWorld="556.697229683717" TargetWaveLengthBegin="548.65512503415" TargetWaveLengthEnd="556.697229683717" DyeName=""/>
              <MultiBand Channel="3" ChannelName="Channel 3" LeftWorld="618.356164383559" RightWorld="700" TargetWaveLengthBegin="618.356164383559" TargetWaveLengthEnd="700" DyeName="Leica/TRITC"/>
              <MultiBand Channel="4" ChannelName="Channel 4" LeftWorld="747.163402828246" RightWorld="799.663402828246" TargetWaveLengthBegin="747.163402828246" TargetWaveLengthEnd="799.663402828246" DyeName=""/>
            </Spectro>
            <AotfList>
              <Aotf Version="0" LightSourceName="SuperContVisible2" LightSourceType="4" OpenVirtual="0" IsChanged="0" ExcitationControlMode="0" ExcitationControlModeBegin="0" ExcitationControlModeEnd="0" CanDoPulseFreq="0" PulsFreq="0" CanDoTwoLaserPIE="0" TwoLaserPIEPossibleLastState="0" TwoLaserPIEActive="0">
                <BeamRoute Version="0">
                  <BeamPosition BeamPositionLevel="0" BeamPosition="10"/>
                  <BeamPosition BeamPositionLevel="1" BeamPosition="1"/>
                </BeamRoute>
                <LaserLineSetting LaserLine="488" IntensityDev="0.499851128208343" IntensityLowDev="0" AOBSIntensityDev="100" AOBSIntensityLowDev="0" EnableDoubleMode="0" LineIndex="0" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="1" OutCheckedIntensity="0" SuppressionMode="1" IsVisible="1" CanDoFastModulation="1"/>
                <LaserLineSetting LaserLine="552" IntensityDev="49.9952060174489" IntensityLowDev="0" AOBSIntensityDev="100" AOBSIntensityLowDev="0" EnableDoubleMode="0" LineIndex="1" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="1" OutCheckedIntensity="0" SuppressionMode="1" IsVisible="1" CanDoFastModulation="1"/>
                <LaserLineSetting LaserLine="472" IntensityDev="0" IntensityLowDev="0" AOBSIntensityDev="-1" AOBSIntensityLowDev="-1" EnableDoubleMode="0" LineIndex="2" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="0" OutCheckedIntensity="0" SuppressionMode="-1" IsVisible="0" CanDoFastModulation="1"/>
                <LaserLineSetting LaserLine="473" IntensityDev="0" IntensityLowDev="0" AOBSIntensityDev="-1" AOBSIntensityLowDev="-1" EnableDoubleMode="0" LineIndex="3" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="0" OutCheckedIntensity="0" SuppressionMode="-1" IsVisible="0" CanDoFastModulation="1"/>
                <LaserLineSetting LaserLine="667" IntensityDev="0" IntensityLowDev="0" AOBSIntensityDev="-1" AOBSIntensityLowDev="-1" EnableDoubleMode="0" LineIndex="4" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="0" OutCheckedIntensity="0" SuppressionMode="-1" IsVisible="0" CanDoFastModulation="1"/>
                <LaserLineSetting LaserLine="668" IntensityDev="0" IntensityLowDev="0" AOBSIntensityDev="-1" AOBSIntensityLowDev="-1" EnableDoubleMode="0" LineIndex="5" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="0" OutCheckedIntensity="0" SuppressionMode="-1" IsVisible="0" CanDoFastModulation="1"/>
                <LaserLineSetting LaserLine="669" IntensityDev="0" IntensityLowDev="0" AOBSIntensityDev="-1" AOBSIntensityLowDev="-1" EnableDoubleMode="0" LineIndex="6" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="0" OutCheckedIntensity="0" SuppressionMode="-1" IsVisible="0" CanDoFastModulation="1"/>
                <LaserLineSetting LaserLine="670" IntensityDev="0" IntensityLowDev="0" AOBSIntensityDev="-1" AOBSIntensityLowDev="-1" EnableDoubleMode="0" LineIndex="7" SequenceIndex="0" LineDeactivationFlags="0" IsLineChecked="0" OutCheckedIntensity="0" SuppressionMode="-1" IsVisible="0" CanDoFastModulation="1"/>
              </Aotf>
            </AotfList>
            <LUT_List>
              <LUT Channel="1" LutName="Green"/>
              <LUT Channel="2" LutName="Red"/>
              <LUT Channel="3" LutName="Red"/>
              <LUT Channel="4" LutName="Gray"/>
              <LUT Channel="100" LutName="Gray"/>
            </LUT_List>
            <DetectorList DetectorAutoSelection="1" InExternalDetectionMode="0">
              <Detector Name="HyD 1" Type="HyD" ScanType="Internal" Channel="1" ChannelName="Channel 1" IsActive="1" Gain="41.5023978574987" Offset="-6.6666666666606E-03" IsSTEDDetector="1" IsSRSDetector="0" DetectionRangeBegin="0" DetectionRangeEnd="0" Phase="0" IsFlimDetector="0" IsHPDDetector="1" IsEnabled="1" IsHPDOverloaded="0" CanDoPhotonCounting="1" AcquisitionMode="0" AcquisitionModeName="PhotonIntegration" CanDoTimeGate="1" IsTimeGateActivated="0" TimeGatePulseStart="300" TimeGateWavelength="488" TimeGatePulseEnd="6000"/>
              <Detector Name="PMT 2" Type="PMT" ScanType="Internal" Channel="2" ChannelName="Channel 2" IsActive="0" Gain="0" Offset="0" IsSTEDDetector="1" IsSRSDetector="0" DetectionRangeBegin="350" DetectionRangeEnd="800" Phase="0" IsFlimDetector="0" IsHPDDetector="0" CanDoTimeGate="0"/>
              <Detector Name="HyD 3" Type="HyD" ScanType="Internal" Channel="3" ChannelName="Channel 3" IsActive="1" Gain="105.842364225212" Offset="-6.6666666666606E-03" IsSTEDDetector="1" IsSRSDetector="0" DetectionRangeBegin="0" DetectionRangeEnd="0" Phase="0" IsFlimDetector="0" IsHPDDetector="1" IsEnabled="1" IsHPDOverloaded="0" CanDoPhotonCounting="1" AcquisitionMode="0" AcquisitionModeName="PhotonIntegration" CanDoTimeGate="1" IsTimeGateActivated="0" TimeGatePulseStart="300" TimeGateWavelength="552" TimeGatePulseEnd="6000"/>
              <Detector Name="PMT 4" Type="PMT" ScanType="Internal" Channel="4" ChannelName="Channel 4" IsActive="0" Gain="0" Offset="0" IsSTEDDetector="1" IsSRSDetector="0" DetectionRangeBegin="350" DetectionRangeEnd="800" Phase="0" IsFlimDetector="0" IsHPDDetector="0" CanDoTimeGate="0"/>
            </DetectorList>
            <LaserArray>
              <Laser Version="6" LightSourceType="4" LaserName="WLL" PowerState="On" StedAlignFlag="1" CanDoLinearOutputPower="1" CanDoPulsing="1" CanDoOutputPowerWatt="1" HighPowerModeActive="0" LightSourceName="SuperContVisible2" OutputPowerWatt="0" OutputPowerPercentage="0.7" Wavelength="0" CanDoChangeWavelength="0">
                <BeamRoute Version="0">
                  <BeamPosition BeamPositionLevel="0" BeamPosition="10"/>
                  <BeamPosition BeamPositionLevel="1" BeamPosition="1"/>
                </BeamRoute>
              </Laser>
            </LaserArray>
            <OnlineDyeSeparation Enabled="0" KeepRawImage="0" Detectors="0" Channels="0" Normalize="0">
              <OnlineDyeSeparationDetectorList/>
              <OnlineDyeSeparationChannelList/>
            </OnlineDyeSeparation>
            <VariableBeamExpanderFactors CommonSettingEnabled="0" CommonFactor="1.03"/>
          </ATLConfocalSettingDefinition>
        </Attachment>
      </Image>
    </Data>
    <Memory Size="655360" MemoryBlockID="MemBlock_76"/>
    <Children/>
  </Element>
nanthony21 commented 3 years ago

No problem, it's definitely not as simple of an issue to resolve as I would have initially guessed. To make matters worse it appears that different versions of Leica's LAS software have completely different ways of formatting the metadata. I ended up having to write a separate XMLMetadata class for each of the two different Leica microscopes that I wanted to use.

I'll paste the code that I used below in case it is useful, but even this only extracts the information that I needed for my particular project. To make a generalized solution that allows access to all available metadata would probably be much more difficult.


class ImageXML(abc.ABC):
    @abc.abstractmethod
    def getHardwareSettings(self) -> t_.Tuple[dict, dict]:
        pass

    @abc.abstractmethod
    def getXYZCoords(self) -> t_.Tuple[float, float, float]:
        pass

    @abc.abstractmethod
    def getVoxelSize(self) -> t_.Tuple[float, float, float]:
        """Returns x,y,z voxel size in microns"""
        pass

    @abc.abstractmethod
    def getPinholeSize(self) -> t_.Tuple[float, float]:
        """Returns the pinhole size. First element is in microns, second is in airy units."""
        pass

class _SP8ImageXML(ImageXML):
    """The XML for older versions of LAS AF appears to be very different from the newer ones but I don't see an easy way to determine version based on the xml.
    Thic class targets the newer format."""
    def __init__(self, xml: ElementTree.Element):
        self._xml = xml

    def getHardwareSettings(self) -> t_.Tuple[dict, dict]:
        el = self._xml.find("Data").find("Image")
        assert el is not None
        for attachment in el.findall("Attachment"):
            if attachment.get("Name") == "HardwareSetting":
                return attachment.find("ATLConfocalSettingDefinition").attrib, None  # The second metadata dictionary just hasn't been implemented here.

    def getXYZCoords(self) -> t_.Tuple[float, float, float]:
        d, _ = self.getHardwareSettings()
        return float(d['StagePosX']), float(d['StagePosY']), float(d['ZPosition'])

    def getVoxelSize(self) -> t_.Tuple[float, float, float]:
        """Returns x,y,z voxel size in microns"""
        raise NotImplementedError()

    def getPinholeSize(self) -> t_.Tuple[float, float]:
        """Returns the pinhole size. First element is in microns, second is in airy units."""
        raise NotImplementedError()

class _SP5ImageXML(ImageXML):
    """The XML for older versions of LAS AF."""
    def __init__(self, xml: ElementTree.Element):
        self._xml = xml

    def getHardwareSettings(self) -> t_.Tuple[dict, dict]:
        """Returns two dictionaries. One is referred to in the metadata as "FilterSettings" The other
         is "ScannerSettings". The distinction is unclear."""
        el = self._xml.find("Data").find("Image")
        assert el is not None
        filterSettings = scannerSettings = None
        for attachment in el.findall("Attachment"):
            if attachment.get("Name") == "HardwareSettingList":
                filterSettings = attachment.find("HardwareSetting").find("FilterSetting")  # A lot of info is saved under "FilterSetting". Not sure why it's called "Filter"
                scannerSettings = attachment.find("HardwareSetting").find("ScannerSetting")  # The `essential` info about the laser scanner is in this element.
                break
        assert filterSettings is not None
        assert scannerSettings is not None
        d = {}
        scannerD = {}
        for setting in filterSettings.findall("FilterSettingRecord"):
            d[setting.attrib['Description']] = setting.attrib['Variant']
        for setting in scannerSettings.findall("ScannerSettingRecord"):
            scannerD[setting.attrib['Identifier']] = setting.attrib['Variant']
        return d, scannerD

    def getXYZCoords(self) -> t_.Tuple[float, float, float]:
        filterSettings, scannerSettings = self.getHardwareSettings()
        return float(filterSettings['DMI6000 Stage Pos x']), float(filterSettings['DMI6000 Stage Pos y']), float(filterSettings['DMI6000 Stage Pos z'])

    def getVoxelSize(self) -> t_.Tuple[float, float, float]:
        """Returns x,y,z voxel size in microns"""
        filterSettings, scannerSettings = self.getHardwareSettings()
        x = float(scannerSettings['dblVoxelX']) * 1e6
        y = float(scannerSettings['dblVoxelY']) * 1e6
        z = float(scannerSettings['dblVoxelZ']) * 1e6
        return x, y, z

    def getPinholeSize(self) -> t_.Tuple[float, float]:
        """Returns the pinhole size. First element is in microns, second is in airy units."""
        filterSettings, scannerSettings = self.getHardwareSettings()
        return float(scannerSettings['dblPinhole']) * 1e6, float(scannerSettings['dblPinholeAiry'])

class LIFFile(LifFile):
    """Adds useful functionality to the file from `readlif`"""
    def getImageXML(self) -> t_.Dict[str, ImageXML]:
        """
        Returns:
            A dictionary of the Root XML element for each image keyed by the image name.
        """
        root = self.xml_root
        if root.find("Element").get("Name") == "Project":  # This is the newer SP8 format of metadata
            xmlClass = _SP8ImageXML
        else:
            xmlClass = _SP5ImageXML
        return {i.attrib['Name']: xmlClass(i) for i in root.find("Element").find("Children").findall("Element")}
nimne commented 3 years ago

No problem, it's definitely not as simple of an issue to resolve as I would have initially guessed. To make matters worse it appears that different versions of Leica's LAS software have completely different ways of formatting the metadata.

While it may never happen.. it would be great if the industry could converge on an open standard. One can dream!

I ended up having to write a separate XMLMetadata class for each of the two different Leica microscopes that I wanted to use.

Thanks for the code block! I think that the approach of extending the class as you did makes more sense than trying to build that functionality into this package - especially if this ends up being a moving target. I don't think I have the bandwidth to get the data read in correctly, and handle metadata changes.

Closing this issue for now.