IndexError: tuple index out of range in ClearMap.Anlysis.Voxelization.voxelizeSphere()

[pforai]

Hi Christoph,

Next hiccup on our end can be seen here:

[...]
Transformix has finished at Wed Jul 13 23:13:36 2016.
Elapsed time: 4 Seconds.

Traceback (most recent call last):
  File "process_template_1.py", line 76, in <module>
    vox = voxelize(points, AtlasFile, **voxelizeParameter);
  File "/groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py", line 45, in voxelize
    data = vox.voxelizeSphere(points.astype('float'), dataSize[0], dataSize[1], dataSize[2], size[0], size[1], size[2]);
IndexError: tuple index out of range

Any idea what to try to debug this?

The voxelization params are

voxelizeParameter = {
     "method" : 'Spherical',
     "size" : (15,15,15),
     "weights" : None,
}

[pforai]

I'm hitting the issue here as in my case dataSize is a 2-tuple but the code always assumes it is a 3-tuple.

(Pdb) n
> /groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py(41)voxelize()
-> elif isinstance(dataSize, basestring):
(Pdb) p dataSize
'/groups/keays/Clearedbrains/MRIBrainsI/ClearMap_ressources/25umAutofluoReference/template_25.tif'
(Pdb) n
> /groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py(42)voxelize()
-> dataSize = io.dataSize(dataSize);
(Pdb) n
> /groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py(44)voxelize()
-> points = io.readPoints(points);
(Pdb) p dataSize
(528, 456)
(Pdb) n
> /groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py(46)voxelize()
-> if method.lower() == 'spherical':
(Pdb) n
> /groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py(47)voxelize()
-> if weights is None:
(Pdb) n
> /groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py(48)voxelize()
-> data = vox.voxelizeSphere(points.astype('float'), dataSize[0], dataSize[1], dataSize[2], size[0], size[1], size[2]);
(Pdb) p points
array([[  67.348145,  152.955466,  198.661709],
       [  68.217184,  156.588074,  198.158196],
       [  68.519498,  159.102957,  196.555896],
       ...,
       [ 145.435807,  217.81448 ,  154.387201],
       [ 144.944172,  220.059512,  152.836961],
       [ 142.952955,  227.738194,  150.920485]])
(Pdb) p size
(15, 15, 15)
(Pdb) n
IndexError: 'tuple index out of range'
> /groups/keays/software/clearmap/lib/python2.7/site-packages/ClearMap-0.9.2-py2.7.egg/ClearMap/Analysis/Voxelization.py(48)voxelize()
-> data = vox.voxelizeSphere(points.astype('float'), dataSize[0], dataSize[1], dataSize[2], size[0], size[1], size[2]);
(Pdb) p dataSize
(528, 456)
(Pdb) p size
(15, 15, 15)
(Pdb)

[ChristophKirst]

The dataSize is assumed to be the size of the final volume and thus naturally must have 3 dimensions. Not sure how you determine its value and why its 2d in your case

[pforai]

As you can see that this was the voxelization for the 25um autoflou reference template TIF that is shipped as part of the resources for the Mouse brain you've got on the page.

The value is determined via dataSize = io.dataSize(dataSize); where dataSize is a string with the value of the path of the 25um template.

Should I use a different template?

[ChristophKirst]

The file we provide for download is a volume and on my machine:

import ClearMap.IO as io fn = 'template_25.tif' io.dataSize(fn)

gives (320, 528, 456)

so in your file / or in the dataSize routine somehow the first dimension is skipped. Have you opened the file or manipulated it ?

Please check the file dimensions with imagej first, if that has the size (320, 528, 456) run the code above. If that gives you a different size please post result here again.

[lukasla]

Following is the info from Fiji the template_25.tif file we used.

NRRD0004
# Complete NRRD file format specification at:
# http://teem.sourceforge.net/nrrd/format.html
type: unsigned short
dimension: 3
space: left-posterior-superior
sizes: 528 320 456
space directions: (25,0,0) (0,25,0) (0,0,25)
kinds: domain domain domain
endian: little
encoding: gzip
space origin: (0,0,0)

------------------------------------------------------
(Fiji Is Just) ImageJ 2.0.0-rc-43/1.50e; Java 1.6.0_65 [64-bit]; Mac OS X 10.10.5; 365MB of 5205MB (7%)

Title: template_25-1.tif
Width:  8000 microns (320)
Height:  13200 microns (528)
Depth:  11400 microns (456)
Size:  147MB
Resolution:  0.04 pixels per micron
Voxel size: 25x25x25 micron^3
ID: -4
Bits per pixel: 16 (unsigned)
Display range: 0 - 516
Image: 210/456
No threshold
Uncalibrated
Path: /Volumes/groups-2/keays/Clearedbrains/MRIBrainsI/ClearMap_ressources/25umAutofluoReference/template_25.tif
Screen location: 820,99 (1280x800)
Coordinate origin:  0,0,0
No overlay
No selection

Running your python code gives us

>>> import ClearMap.IO as io
>>> fn = 'template_25.tif'
>>> fn = '/groups/keays/Clearedbrains/MRIBrainsI/ClearMap_ressources/template_25.tif' 
>>> io.dataSize(fn)
(528, 456)
>>> 

Are 3D TIFFs some extension that libtiff or Python tifffile doesnt support?

[ChristophKirst]

puzzling.

Just to be sure, in the python code you use: '/groups/keays/Clearedbrains/MRIBrainsI/ClearMap_ressources/template_25.tif' for the fiji you use: '/Volumes/groups-2/keays/Clearedbrains/MRIBrainsI/ClearMap_ressources/25umAutofluoReference/template_25.tif'

which looks like you are using two different files here possibly?

the tifffile module is a bit of a struggle sometimes, but i dont have the problem on my machine, which versions of tiffile are you using ?

try

>>> import tifffile
>>> tifffile.__version__

[lukasla]

This is just a path mismatch between the HPC cluster and the local Mac, but is the same file. Thanks for noting.

>>> import tifffile
>>> print tifffile.__version__
0.9.2
>>> 

[ChristophKirst]

ok, i am using: '0.6.2'

could you try that older version and see if you still get the same error, i look will into using the 0.9.2 version

[ChristophKirst]

also could you run this on the file:

>>> import tifffile as tiff
t = tiff.TiffFile(filename);
print len(t.pages)
print t.pages[0].shape

thanks

[lukasla]

Sure thing:

>>> t = tiff.TiffFile(fn);
>>> print len(t.pages)
1
>>> print t.pages[0].shape
(456, 528, 320)

[ChristophKirst]

seems that the new tifffile version sees the 3d data but handles it slightly differently, let me fix this

[ChristophKirst]

try pulling the new version, it should now work with the new version 0.9.2 of tifffile.