Java package 'loci.formats' has no attribute 'in'

[humbertojoca]

I was trying to use Bioformats reader but I getting the following error:

reader = pims.bioformats.BioformatsReader(file)

File "C:\ProgramData\Anaconda3\lib\site-packages\pims\bioformats.py", line 364, in init mo = getattr(loci.formats, 'in').DynamicMetadataOptions()

AttributeError: Java package 'loci.formats' has no attribute 'in'

I already have jpype and the loci_tools.jar is on the pims folder. Am I missing something?

[joaomamede]

I have the same with .nd2 files .ome.tif files and .dv files.

I think this was a regression as it was an old problem that was fixed before? Example:

 pims.open("trytostitch.nd2")
/home/jmamede/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/bioformats.py:54: UserWarning: loci_tools.jar not found, downloading
  warn('loci_tools.jar not found, downloading')
/home/jmamede/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/api.py:207: UserWarning: <class 'pims.bioformats.BioformatsReader'> errored: Java package 'loci.formats' has no attribute 'in'
  warn(message)
---------------------------------------------------------------------------
UnknownFormatError                        Traceback (most recent call last)
<ipython-input-3-7dd390e47616> in <module>
----> 1 pims.open("trytostitch.nd2")

~/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/api.py in open(sequence, **kwargs)
    207             warn(message)
    208             exceptions += message + '\n'
--> 209     raise UnknownFormatError("All handlers returned exceptions:\n" + exceptions)
    210 
    211 

UnknownFormatError: All handlers returned exceptions:
<class 'pims.bioformats.BioformatsReader'> errored: Java package 'loci.formats' has no attribute 'in'

reader = pims.bioformats.BioformatsReader("trytostitch.nd2")
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-4-b4e15f334554> in <module>
----> 1 reader = pims.bioformats.BioformatsReader("trytostitch.nd2")

~/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/bioformats.py in __init__(self, filename, meta, java_memory, read_mode, series)
    362         # See https://github.com/openmicroscopy/bioformats/issues/2955
    363         # circumventing the reserved keyword 'in'
--> 364         mo = getattr(loci.formats, 'in').DynamicMetadataOptions()
    365         mo.set('nativend2.chunkmap', 'False')  # Format Bool as String
    366         self.rdr.setMetadataOptions(mo)

AttributeError: Java package 'loci.formats' has no attribute 'in'

In [7]: pims.open("B01.nd2 - B01.nd2 (series 01) - C=0.tif")
Out[7]: 
<Frames>
Source: B01.nd2 - B01.nd2 (series 01) - C=0.tif
Length: 39 frames
Frame Shape: (2044, 2048)
Pixel Datatype: uint16

In [8]: pims.open("trytostitch_13.ome.tif")
ImageJ series: invalid metadata or corrupted file
Out[8]: 
<Frames>
Source: trytostitch_13.ome.tif
Length: 53 frames
Frame Shape: (2044, 2048)
Pixel Datatype: uint16

In [9]: reader = pims.bioformats.BioformatsReader("trytostitch_13.ome.tif")
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-9-def637b8aaf6> in <module>
----> 1 reader = pims.bioformats.BioformatsReader("trytostitch_13.ome.tif")

~/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/bioformats.py in __init__(self, filename, meta, java_memory, read_mode, series)
    362         # See https://github.com/openmicroscopy/bioformats/issues/2955
    363         # circumventing the reserved keyword 'in'
--> 364         mo = getattr(loci.formats, 'in').DynamicMetadataOptions()
    365         mo.set('nativend2.chunkmap', 'False')  # Format Bool as String
    366         self.rdr.setMetadataOptions(mo)

AttributeError: Java package 'loci.formats' has no attribute 'in'

In [14]: pims.bioformats.BioformatsReader('SA_U_02D117_Vag3_CD4.Cy5_CD163.RedX_ECAD.FI
    ...: TC_DAPI_SL16_20x1280_01_P1_R3D_D3D_PRJ.dv')
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-14-0c412901094f> in <module>
----> 1 pims.bioformats.BioformatsReader('SA_U_02D117_Vag3_CD4.Cy5_CD163.RedX_ECAD.FITC_DAPI_SL16_20x1280_01_P1_R3D_D3D_PRJ.dv')

~/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/bioformats.py in __init__(self, filename, meta, java_memory, read_mode, series)
    362         # See https://github.com/openmicroscopy/bioformats/issues/2955
    363         # circumventing the reserved keyword 'in'
--> 364         mo = getattr(loci.formats, 'in').DynamicMetadataOptions()
    365         mo.set('nativend2.chunkmap', 'False')  # Format Bool as String
    366         self.rdr.setMetadataOptions(mo)

AttributeError: Java package 'loci.formats' has no attribute 'in'

In [15]: 

[joaomamede]

It seems that it might be a java bridge failure, which pims usually starts for us.

https://github.com/LeeKamentsky/python-javabridge/issues/88

[Thrameos]

Assuming you are using JPype and not one of the other bridge codes, then anything matching a keyword in Python would be renamed with a trailing underscore (assuming everything is correct in the keywords list.)

Example: Java

public class Test
{
        public int in = 5;
}

Python:

import jpype
jpype.startJVM(classpath='.')

t=jpype.JClass('Test')()
print(t.in_)  # The symbol was renamed due to keyword conflicts

Testing it prints "5" so it works as expected.

JPype does not keep the old definition with conflict as you wouldn't be able to access it using normal syntax. I suppose it could in this sort of case but it would be a lot of additional logic for little gain.

[Thrameos]

Oh if in the off chance it is a package name rather than a class field, the same rules apply.

If we create a class called org.in.Test, we can try to import it.

import jpype
import jpype.imports
jpype.startJVM(classpath='pkgtest/classes')

# If we use Python syntax to access it the we have to use the mangled name.
import org.in_ as q
print(dir(q))  # prints ["Test"]

# On the other hand we can also request a direct access to the JPackage which will not use mangled names.
p = jpype.JPackage('org.in')
print(dir(p))  # prints ["Test"]

Please note that the attribute in org will be "in_" as all attribute names are mangled automatically on conflicts.

See https://jpype.readthedocs.io/en/latest/quickguide.html#id24 for further Java examples.

[joaomamede]

I really can't pinppoint the problem I have an old environment in anaconda that loads the file with no problem.

The one that I just created (and pims did fetch right now has the error). Also, if I copy the loci_tools.jar to the new environment..It does not work.

Pims 0.5 and jpype 1.1.2 (in all environments).

Older environment (my base on anaconda) version:

In [1]: import pims

In [2]:  a = pims.open('./trytostitch.nd2')
/home/jmamede/anaconda3/lib/python3.7/site-packages/pims/base_frames.py:472: UserWarning: Please call FramesSequenceND.__init__() at the start of thethe reader initialization.
  warn("Please call FramesSequenceND.__init__() at the start of the"

In [3]: a
Out[3]: 
<FramesSequenceND>
Axes: 6
Axis 'x' size: 2048
Axis 'y' size: 2044
Axis 'c' size: 3
Axis 't' size: 1
Axis 'm' size: 30
Axis 'z' size: 25
Pixel Datatype: <class 'numpy.uint16'>

In [4]: pims.__version__
Out[4]: '0.5'

Fresh environment:

In [1]: import pims

In [2]:  a = pims.open('./trytostitch.nd2')
/home/jmamede/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/api.py:207: UserWarning: <class 'pims.bioformats.BioformatsReader'> errored: Java package 'loci.formats' has no attribute 'in'
  warn(message)
------------------------------------------------------------------------
UnknownFormatError                     Traceback (most recent call last)
<ipython-input-2-7cc039e2e386> in <module>
----> 1 a = pims.open('./trytostitch.nd2')

~/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/api.py in open(sequence, **kwargs)
    207             warn(message)
    208             exceptions += message + '\n'
--> 209     raise UnknownFormatError("All handlers returned exceptions:\n" + exceptions)
    210 
    211 

UnknownFormatError: All handlers returned exceptions:
<class 'pims.bioformats.BioformatsReader'> errored: Java package 'loci.formats' has no attribute 'in'

In [3]: pims.__version__
Out[3]: '0.5'

After copying my old loci_tools.jar to the new environment:

In [1]: import pims

In [2]:  a = pims.open('./trytostitch.nd2')
/home/jmamede/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/api.py:207: UserWarning: <class 'pims.bioformats.BioformatsReader'> errored: Java package 'loci.formats' has no attribute 'in'
  warn(message)
---------------------------------------------------------------------------
UnknownFormatError                        Traceback (most recent call last)
<ipython-input-2-7cc039e2e386> in <module>
----> 1 a = pims.open('./trytostitch.nd2')

~/anaconda3/envs/pycuda/lib/python3.7/site-packages/pims/api.py in open(sequence, **kwargs)
    207             warn(message)
    208             exceptions += message + '\n'
--> 209     raise UnknownFormatError("All handlers returned exceptions:\n" + exceptions)
    210 
    211 

UnknownFormatError: All handlers returned exceptions:
<class 'pims.bioformats.BioformatsReader'> errored: Java package 'loci.formats' has no attribute 'in'

In [3]: pims.__version__
Out[3]: '0.5'

[joaomamede]

I just commented this on bioformats.py from pims.

        # patch for issue with ND2 files and the Chunkmap implemented in 5.4.0
        # See https://github.com/openmicroscopy/bioformats/issues/2955
        # circumventing the reserved keyword 'in'
        mo = getattr(loci.formats, 'in').DynamicMetadataOptions()
        mo.set('nativend2.chunkmap', 'False')  # Format Bool as String
        self.rdr.setMetadataOptions(mo)

and it reads any file without a problem now.

I tested .nd2 (nikon), dv (deltavision), ome.tiff (generated by me).

When I tried previously manually it just spitted the same : there's no "in" class.

[joaomamede]

Adding the in_ works:

        mo = getattr(loci.formats, 'in_').DynamicMetadataOptions()
        mo.set('nativend2.chunkmap', 'False')  # Format Bool as String
        self.rdr.setMetadataOptions(mo)

That fixes it.

Edit: My old base environment had 'in_' already, I guess this was a regression where the underscore got removed?

[Thrameos]

It should just be loci.formats.in_.DynamicMeradataOptions() . Some previous version must have been for pre1.0 were there was still a bug in the rename system.