abria / TeraStitcher

A tool for fast automatic 3D-stitching of teravoxel-sized microscopy images
http://abria.github.io/TeraStitcher/
Other
78 stars 32 forks source link

Expected throughput for teraconverter with hdf5 (imaris) output #92

Open richardbeare opened 2 years ago

richardbeare commented 2 years ago

Hi, How long should I expect the stitching phase to take for a 100GB + imaris file? I have tested a couple of configurations after the step failed to complete in 72 hours on a cluster job. The cluster is configured with a filesystem configured for good throughout of large files (for genetics work). I suspected that the IO of teraconverter may use lots of smaller IO requests, so I'm now running a test with output to a local filesystem. The system is a fairly typical linux desktop and I see an approximate 20GB increase in output file size in 12 hours. Is this typical? What should I look at to speed it up?

FYI - I have had no success getting the MPI tools to work with hdf5/imaris input or output files.

Thanks