ITK-Snap crashed when opening large datasets. It seems this was caused by an integer overflow when calculating a pointer index. The function used dot_product from VNL, which does not check for overflows (http://itk.org/gitweb?p=ITK.git;a=blob;f=Modules/ThirdParty/VNL/src/vxl/core/vnl/vnl_sse.h;h=04f930ef05bb49036e1fa69d6fe9ae9a697e2d45;hb=HEAD#l189). Doing the calculation avoiding VNL allows to use a simple cast to size_t before multiplication.
This solved the issue for me. I can now load a grey image of about 10GB plus a 40GB segmentation into ITK-Snap, modify it (even in gWS mode) and then render the segmentation without any problems.
Perhaps using size_t for the vectors used for index calculation (instead of Vector3i) might be a cleaner approach. I'm not sure, but the casting solution might slow down ITK-Snap and is only needed for big datasets. Perhaps a pre-check for overflow could be introduced and dot_product used as long as possible.
I'm not sure if it is a good idea to encourage users to use ITK-Snap for such big datasets. If they shouldn't, ITK-Snap should at least through an exception that the dataset to be loaded is too big instead of crash.
ITK-Snap crashed when opening large datasets. It seems this was caused by an integer overflow when calculating a pointer index. The function used dot_product from VNL, which does not check for overflows (http://itk.org/gitweb?p=ITK.git;a=blob;f=Modules/ThirdParty/VNL/src/vxl/core/vnl/vnl_sse.h;h=04f930ef05bb49036e1fa69d6fe9ae9a697e2d45;hb=HEAD#l189). Doing the calculation avoiding VNL allows to use a simple cast to size_t before multiplication. This solved the issue for me. I can now load a grey image of about 10GB plus a 40GB segmentation into ITK-Snap, modify it (even in gWS mode) and then render the segmentation without any problems. Perhaps using size_t for the vectors used for index calculation (instead of Vector3i) might be a cleaner approach. I'm not sure, but the casting solution might slow down ITK-Snap and is only needed for big datasets. Perhaps a pre-check for overflow could be introduced and dot_product used as long as possible. I'm not sure if it is a good idea to encourage users to use ITK-Snap for such big datasets. If they shouldn't, ITK-Snap should at least through an exception that the dataset to be loaded is too big instead of crash.