Building an annotation tool - Help needed

[indahaus]

Hello Everyone,

My name is Nebojša and I'm a master student of Mechatronics at Hochschule Karlsruhe. As a project at HKA in the summer semester, my colleague and I need to build an annotation tool for 3D scans from an industrial CT scanner. Not only does it need to process large data (images are over 25GB in size), but it also needs to be efficient in terms of resources and time.

Through our research, we discovered cLesperanto (in napari), and found it extremely useful as it prioritizes GPU over CPU. However, we are facing a problem: it seems to lack region-growing algorithms. These algorithms are crucial for us because our idea is to plant a seed in a defect (pore or crack), which is sometimes just a couple of pixels/voxels wide/deep, and annotate the whole defected region. Thresholding algorithms in this application are not suitable, and the parameters are hard to identify and tune.

I should mention that we know almost nothing about image processing with its methods and terminology, and our programming knowledge is very limited, so please forgive me if I seem inexperienced.

Given this, I was wondering if you could help us by providing some rough guidelines on how we could approach our task and what we could possibly do to engage more with the topic and specific requirements of the project. If necessary, I could provide you with additional information and datasets.

P.S. I have contacted mr. Haase directly over email and he instructed me to open a topic here. Also, he mentioned that I should explicitly list the algorithms (citations) I'm interested in, but I am unsure (due to my lack of knowledge) which algorithms I should address. As I briefly mentioned, we think that the Region Growing algorithms would suit our application the best, but maybe there are some other (maybe better) algorithms which are we unaware of.

This is the image of one slice of the given scan:

This image shows cropped area of one of the slices, with intensified contrast. Our tool would need to segment these black spots (defects) in 3D.

[StRigaud]

Hello @indahaus

Indeed, clesperanto do not have region-growing approaches (yet). It's something we could try to add to our library in the future. Although, it might arrive a bit late for your project. And if you have a specific implementation in mind that could help us.

our idea is to plant a seed in a defect (pore or crack), which is sometimes just a couple of pixels/voxels wide/deep, and annotate the whole defected region.

I could suggest a marker-controled voronoi segmentation:

1. place manually a set of markers (seeds)
2. extract a binary mask of the darker areas in your image
3. and use masked_voronoi_labeling() to extend the seeds inside the delimittation of the mask

The binary mask may be the main issue, and you may need to use a bit of preprocessing like a top_hat and a median filter to get the defects. An other approach is a difference of gaussian which should return high value where the defects are. In both approach there will be some fine-tuning and testing.

A good side is that if you are in an industrial context, you can expect little to no variation between image acquisition (hopefully). So parameters calibration should not change too much between images.

@haesleinhuepf if you have other ideas, feel free :)

images are over 25GB in size

FYI, understand that your GPUs may not be adapted to images of this size. GPU memories have several limitation in maximal object size in memory as well as the amount of possible memory (e.g. NVIDIA RTX 4090 have 32Gb) You may not be able to process you data in one run and will have to process it in chunk or per slice.

[haesleinhuepf]

Hi @indahaus and @StRigaud ,

great discussion here! I'm just adding links and comments to Stephane's excellent advice:

• Voronoi-Otsu-Labeling could potentially work on the inverted image.
• Tiled image processing might indeed be key for processing such large data.

Question for @StRigaud: Might Morphological Chan-Vese work in this context?

Question @indahaus :

region-growing algorithms. These algorithms are crucial for us

Which algorithms are you referring to? If you can provide a link to an algorithm, we can tell you if it is possible/reasonable to do this on a GPU. Without this reference it is pretty much impossible to help you.

[StRigaud]

Might https://github.com/clEsperanto/pyclesperanto_prototype/pull/304 work in this context?

It could but it is greedy in memory (5-6 time the image memory). Also I just pushed it CLIc, I would still like to do some benchmark, tests, and improvement (if needed).

[indahaus]

I could suggest a marker-controled voronoi segmentation:

1. place manually a set of markers (seeds)
2. extract a binary mask of the darker areas in your image
3. and use masked_voronoi_labeling() to extend the seeds inside the delimittation of the mask

I did as suggested, but I'm not sure if I did it correctly, here are the results:

---

The binary mask may be the main issue, and you may need to use a bit of preprocessing like a top_hat and a median filter to get the defects. An other approach is a difference of gaussian which should return high value where the defects are. In both approach there will be some fine-tuning and testing.

I tried these preprocessing methods, difference of gaussian seems to be effective, but I am not sure how to tune parameteres. Should I always use the same value for every direction? I tried these:

Top-hat provided a confusing result:

---

FYI, understand that your GPUs may not be adapted to images of this size. GPU memories have several limitation in maximal object size in memory as well as the amount of possible memory (e.g. NVIDIA RTX 4090 have 32Gb) You may not be able to process you data in one run and will have to process it in chunk or per slice.

Thanks a lot! Also, is it possible to load it into RAM instead? In the lab we have a workstation with 64GB of RAM, so if that is possible, it might be a workaround.

---

@haesleinhuepf Thanks for the links provided! When it comes to Region Growing, my colleague was focusing on MITK, there he found a tool that does what we might need, but the problem with MITK (due to my understanding) is that it's too slow and not capabe of processing large data. So I was wondering if we could achieve the same or similar in napari using clE. Here is what he found:

[StRigaud]

I did as suggested, but I'm not sure if I did it correctly, here are the results:

You should invert the threshold mask that you make so that it is the defect that are segmented, not your object. Threshold detect bright object, here you structure of interest are dark.

I tried these preprocessing methods, difference of gaussian seems to be effective, but I am not sure how to tune parameteres.

Same here, your object of interest is dark, you need to inverse your image as filters are, by default, focussing on high intensity, not low. I understood that you do not have a formation in Image Processing but I encourage you to read a few introduction courses or explaination on the filter. And to test the filters with various parameter to understand their effect.

is it possible to load it into RAM instead?

Not if you want to use your GPU. CPU and GPU have separated memory. The only workaround is to split the data into smaller problematic.