MultiCut Watershed Error

[glancybp]

I am trying to use the Multicut workflow on a large 3D dataset and I keep getting the below error after several hours of attempting to calculate the superpixels. Does anyone know a way to fix this or work around it?

Invariant violation! connected components: Need more labels than can be represented in the destination type. (c:\conda3\conda-bld\vigra_1513957436976\work\include\vigra\union_find.hxx:284)

Does this mean there are too many superpixels to fit within the bit type? I've tried switching to clustered seed labeling and increasing the min superpixel size to 1000 pixels and still get this error.

I am using the Ilastik 1.3.0 GUI on a Windows PC with 2 TB of RAM. The raw dataset is 2839x2333x2263 8bit pixels and I am using an 8 bit probabilities file from the pixel classification workflow. This error happens whether I attempt to calculate the superpixels within the multi workflow or through batch processing. I've used the Multicut workflow successfully with many smaller datasets (up to 200x2333x3956 on Windows PC with 256 Gb RAM) but am trying to scale up. Any help would be appreciated. Thanks.

[DerThorsten]

The following could be going on:

• union_find starts by each pixel having its own label.
• in the next steps these labels are merged to form connected components.
• I assume uint32 is used as datatype for the python bindings of union_find for a volume of 2839x2333x2263 with 14988724781 pixels a data type larger than uint32 is needed ( max val 4294967295).

[glancybp]

That sounds like a major limitation to using MultiCut for large scale analyses. However, I'm not sure if that is the cause here since, theoretically, the error wouldn't happen until a dataset larger than 1625x1625x1625 is used. I got the same error when I attempted the watershed on a 1500x1500x1500 dataset (but not 1000x1000x1000), which should fit within the uint32 pixel limit.

Regardless, I was able to workaround the problem by creating the superpixels using the general MultiCut watershed process, but in ImageJ using the MorphoLibJ (for extended maxima) and 3D ImageJ Suite (for watershed) plugins after creating a smoothed distance transform of the filtered, binarized boundary probabilities. After loading the superpixels back into the MultiCut workflow, I was able to perform the multicut successfully.

For reference, the total number of superpixels within my dataset ended up being 14288. I don't know what's different about the ImageJ plugin approach that worked compared to the MultiCut approach that didn't, but it might be worth looking into if Ilastik is to be used for large scale connectivity analyses.