Parastitcher problem (float division by zero)

[snehashis-roy]

Hello. I want to run the merge step with parastitcher. My specs are,

python 2.7.12
TeraStitcher-portable-1.11.6-Linux
OpenMPI 1.10.7 (libmpi.so.12.0.7)
mpi4py 3.0.3

I have downloaded the parastitcher.py RELEASE 3.2.3.

This is how I normally run the step 6,

TeraStitcher-portable-1.11.6-Linux/terastitcher --merge --projin=parastitcher/result/placetiles.xml --volout=parastitcher/result --volout_plugin="TiledXY|2Dseries" --resolutions=0 --slicewidth=-1 --sliceheight=-1 --slicedepth=-1 --imin_channel=all --imout_plugin=tiff2D --imout_format=tif --imout_depth=16 --stitchablesonly - --algorithm=SINBLEND

This is how I tried to use the parastitcher.py

mpirun -np 6 python TeraStitcher-portable-1.11.6-Linux/parastitcher.py --merge --projin=parastitcher/result/placetiles.xml --volout=parastitcher/result --volout_plugin="TiledXY|2Dseries" --resolutions=0 --slicewidth=-1 --sliceheight=-1 --slicedepth=-1 --imin_channel=all --imout_plugin=tiff2D --imout_format=tif --imout_depth=16 --stitchablesonly - --algorithm=SINBLEND

This is the error that I got,

('The value for ', '--info', ' was not declared. It will be set to', 'no_info', 'by default.')
('The value for ', '--info', ' was not declared. It will be set to', 'no_info', 'by default.')
('The value for ', '--info', ' was not declared. It will be set to', 'no_info', 'by default.')
('The value for ', '--info', ' was not declared. It will be set to', 'no_info', 'by default.')
('The value for ', '--info', ' was not declared. It will be set to', 'no_info', 'by default.')
('The value for ', '--info', ' was not declared. It will be set to', 'no_info', 'by default.')
which: no teraconverter in (.)
which: no teraconverter in (.)
which: no teraconverter in (.)
which: no teraconverter in (.)
which: no teraconverter in (.)
which: no teraconverter in (.)
/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/teraconverter
/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/teraconverter
/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/teraconverter
/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/teraconverter
/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/teraconverter
/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/teraconverter
********************************************************************************
('2020-10-24 20:27:25.878190', ' -- Calculation started on ', 6, '- 1 cores.')
********************************************************************************
teraconverter  --sfmt="TIFF (unstitched, 3D)" --dfmt="TIFF (tiled, 3D)" -s="parastitcher/result/placetiles.xml" -d=/ --info="/home/roys5/roys5/inhomogeneity_correction/__dims__.txt"
('The value for ', '--origin=', ' was not declared. It will be set to', '/home/roys5/roys5/inhomogeneity_correction/__dims__.txt', 'by default.')
('Origin file is: ', '/home/roys5/roys5/inhomogeneity_correction/__dims__.txt')
('The value for ', '--isotropic', ' was not declared. It will be set to', 'False', 'by default.')
('vxl_V, vxl_H, vxl_D, isotropic, h, max_res, max_res_D :', 1.625, 1.625, 2.0, False, 0, 0, 0)
################################################################################
('Input file = ', 'parastitcher/result/placetiles.xml')
('Output directory', 'parastitcher/result')
('Rough depth for the tiles in width direction = ', -1)
('Rough depth for the tiles in height direction = ', -1)
('Rough depth for the tiles in depth direction = ', -1)
('Source Format = ', '"TIFF (unstitched, 3D)"')
('Destination Format = ', '"TIFF (tiled, 2D)"')
('Resolutions = ', [0])
('Max Resolutions', 0)
('Width (in voxel) of the immage = ', 6092)
('Height (in voxel) of the immage = ', 9456)
('Depth (in voxel) of the immage = ', 2442)
['--imin_channel=all', '--imout_plugin=tiff2D', '--imout_format=tif', '--imout_depth=16', '--stitchablesonly', '-', '--algorithm=SINBLEND']
('Last input elements of the original string = ', '--imin_channel=all --imout_plugin=tiff2D --imout_format=tif --imout_depth=16 --stitchablesonly - --algorithm=SINBLEND')
################################################################################
Traceback (most recent call last):
  File "/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/parastitcher.py", line 1678, in <module>
    (first_string, list_string, output_name, len_arr, final_string) = create_commands(nprocs-1) # 2017-02-06. Giulio. @ADDED output_name
  File "/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/parastitcher.py", line 1304, in create_commands
    size_1 = create_sizes(depth, wb1, max_res_D) # max_res_D = max_res if isotropic is not set of if voxels are isotropic
  File "/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/parastitcher.py", line 1201, in create_sizes
    values = opt_algo(size, wb, max_res)
  File "/home/roys5/roys5/Programs/TeraStitcher-portable-1.11.6-Linux/parastitcher.py", line 1153, in opt_algo
    k_h = floor(D/(pow(2.,n) * b_h));
ZeroDivisionError: float division by zero

The merge step with terastitcher successfully works though. The xml files generated before the merge step are attaches.

xmls.zip

My images are saved as 4x3 mosaic, where each tile is a 3D image of size 2560x2160x2502 (HxWxD)

Please let me know how to run the Parastitcher correctly.

[iannellog]

Since the merge step must write output files on disk, to parallelize execution you must generate an output image divided in multiple files, so as each process write a different file to disk (see our paper on parallelization for more details). Since your output format is a series of 2D images it is sufficient that you give a positive value to option --slicedepth. I suggest a value of 1 if your image is large in X-Y, or a relatively small integer if the image is not so large in X-Y, but consists of many slices. Consider also that current version of parastitcher does not accepts options: --imin_channel --imout_format --imout_depth --stitchablesonly (defaults should work well in your case).

[snehashis-roy]

Since the merge step must write output files on disk, to parallelize execution you must generate an output image divided in multiple files, so as each process write a different file to disk (see our paper on parallelization for more details). Since your output format is a series of 2D images it is sufficient that you give a positive value to option --slicedepth. I suggest a value of 1 if your image is large in X-Y, or a relatively small integer if the image is not so large in X-Y, but consists of many slices. Consider also that current version of parastitcher does not accepts options: --imin_channel --imout_format --imout_depth --stitchablesonly (defaults should work well in your case).

Thank you. So in my case, the arguments should be this.

mpirun -np 6 python parastitcher.py --merge --projin=parastitcher/result/placetiles.xml 
--volout=parastitcher/result --volout_plugin="TiledXY|2Dseries" --resolutions=0 
--slicewidth=6092 --sliceheight=9456 --slicedepth=1 --imout_plugin=tiff2D   --algorithm=SINBLEND

Is there a way to know what the stitched image size would be before running this merge step? I can not find the stitched image height/width (i.e. 6092 and 9456) in the placetiles.xml.

[iannellog]

Right. You may specify -1 for options --slicewidth and --sliceheight. You can know the image size running teraconverter with option --info=file_name and other needed options (i.e. -s, --sfmt, --dfmt, the last one useless in this case, but nevertheless required) in file_name you will find the following information (values are dummy): HEIGHT=1892 WIDTH=1892 DEPTH=5 BYTESxCHAN=2 DIM_C=1 VXL_V=0.520000 VXL_H=0.520000 VXL_D=1.000000

[snehashis-roy]

Thank you. This worked out nicely.

Is there any possibility for the Parastitcher that is compatible with Python 3?

[iannellog]

Yes, the current script should work also with Python3. Let me know if you have any problems.

[snehashis-roy]

Yes, the current script should work also with Python3. Let me know if you have any problems.

Thank you. It worked with Python 3.6.10.

[austinhoag]

Right. You may specify -1 for options --slicewidth and --sliceheight.

This method did not work for me with parastitcher 3.2.3, terastitcher 1.11.10. I get the error that @piby2 originally got above:

ZeroDivisionError: float division by zero

when using --sliceheight=-1 --slicewidth=-1 --slicedepth=1.

or can know the image size running teraconverter with option --info=file_name and other needed options (i.e. -s, --sfmt, --dfmt, the last one useless in this case, but nevertheless required)

This also did not work for me, but I'm not as familiar with teraconverter so this could be down to an improper setup on my end. The first time I tried to run this was after running the first 5 steps of terastitcher (i.e. up to Align but before merge). At this point my input folder looked like this:

├── 114510
├── 148100
├── 181690
├── mdata.bin
├── xml_displcomp.xml
├── xml_displproj.xml
├── xml_displthres.xml
├── xml_import.xml
└── xml_placetiles.xml

where I am using the two-level file hierarchy. When I tried to run teraconverter to determine the width and height dimensions of the stitched volume:

teraconverter -s="/jukebox/wang/ahoag/test_stitching_mpi/data" --sfmt="TIFF (tiled, 2D)" --dfmt="TIFF (series, 2D)" -d="/jukebox/wang/ahoag/test_stitching_mpi/stitched"

I got this error:

ERROR: in Stack::unBinarizeFrom(...): error while reading binary metadata file

I moved the mdata.bin file in my folder data somewhere else and re-ran the same command and got this error:

ERROR: in StackedVolume::StackedVolume(...): unable to find metadata file at /jukebox/wang/ahoag/test_stitching_mpi/data/mdata.bin

The whole point for me doing all this is to identify the dimensions of my stitched data before running terastitcher at all so that I can set up the script ahead of time to run all terastitcher steps in a dependency slurm chain. However, I found that when you run the merge step with parastitcher you can just pass --slicewidth and --sliceheight parameters that are very much larger than what you know the real dimensions will be. If you do this, the code will end up using the correct dimensions. For example, I have a dataset that will turn out to be 7382x5642x3540 (I know this from manually running the merge step as a test). I am able to run the merge step and have it produce complete z planes without chunking them up in x and y and without knowing the output dimensions ahead of time by doing this:

mpirun -np 9 parastitcher.py -6 --projin=/jukebox/wang/ahoag/test_stitching_mpi/da/xml_placetiles.xml --volout=/jukebox/wang/ahoag/test_stitching_mpi/stitched --resolutions=0 --sliceheight=20000 --slicewidth=20000 --slicedepth=1