Add morphological ops

[ctrueden]

This includes dilate, erode, open and close.

And we want them to work on arbitrary structuring elements.

[ctrueden]

From Gabriel Landini:

Not sure if this is relevant but just in case... if there are any plans to change the code of the erosion operator from the current IJ algorithm, please can say that it is extremely important not to change or lose the Binary Option of "pad edges" that allows or not to erode image borders. I.e., for a completely filled image, if you apply an erosion, you could consider that the border of the image is the end of the universe (and it gets eroded, as outside it is 0), or you can asume that this is a window in a much larger universe (and so you assume it is filled) and remains not eroded. Some might not worry too much about this detail, but there are good reasons for dealing with those distinct 2 cases, it would break a large amount of our code if that is changed.

[tinevez]

Dear all, In the past I have been working on building some morphological operators n imglib2. The WPI can be found in the morphologyops (outdated) branch of the imglib2, forked before imglib2 github repo was split in sub-repos. An example file from last year: https://github.com/imglib/imglib2/blob/morphologyops/algorithms/core/src/main/java/net/imglib2/algorithm/morphology/MorphologicalOperations.java

I am willing to resume this work, but wanted to ask first:

• Is it fine to push this to imglib2-algorithm?
• Or should it be done in imagej-ops?
• Or should I not do it at all?

[dietzc]

Hi @tinevez,

I would somehow use imagej-ops for this implementation, due to the following reasons: If you use ImageJ-Ops, you will have all the benefits (multi-threading, replace implementationsreplacable, directly usable in combination with other existing ops etc).

On ImgLib2-Side everything related to Shape, Neighborhood etc should be used or extended. For example arbitrary structuring elements (i.e. shapes) are one open issue which we will tackle at the upcoming hackathon when it comes to ROIs (correct me if I'm wrong @tpietzsch ).

On Ops side you can already reuse some existing operations, for example https://github.com/imagej/imagej-ops/blob/master/src/main/java/net/imagej/ops/neighborhood/MapNeighborhood.java. The TODOs in the operations seem to be not valid anymore, as each neighborhood is automatically run in parallel since internally the ParallelMap is used.

The cool thing is, if you use MapNeighborhood, your MorphOperation will automatically benefit if someone writes e.g. a super fast version of MapNeighborhood for 2d ArrayImgs (as an example).

To conclude my suggestions here (and again, @tpietzsch @ctrueden @hinerm etc) may correct me if I'm wrong: Everything related to Neighborhood and Shape should really be done in imglib2. The MorphOp itself should live in imagej-ops.

Btw @iarganda also wanted to implement some MorphOps. Maybe you can join forces here.

[ctrueden]

@tinevez I added you to the OPS developer team on GitHub. Please let us know if you need anything else to support this work.

[ctrueden]

@iarganda I invited you too.

[tinevez]

Hi all. Thank for all of this. I probably overlooked the problem a bit: last year I started an imglib2 package that could do morp-ops. I am porting it back to modern imglib2, but this will take me time. You can find it here: https://github.com/tinevez/imglib2-algorithm/tree/morphology

WARNING DOES NOT COMPILE YET It relies on the Shape interface by @tpietzsch for the structuring elements, so right now this part will live in imglib2-algorithm.

After this, there are two classes that offer static utils

• https://github.com/tinevez/imglib2-algorithm/blob/morphology/src/main/java/net/imglib2/algorithm/morphology/StructuringElements.java generate the (potentially optimized) structurong elements
• https://github.com/tinevez/imglib2-algorithm/blob/morphology/src/main/java/net/imglib2/algorithm/morphology/MorphologicalOperations.java uses the above strels to perform the actual morphological operations. Right now, not all of them are implemented.

I guess the imagej-ops class can be based on the imglib2 classes rather than the opposite, no?

[dietzc]

All the Shape and Neighborhood should really live in imglib2-ROI or imglib2-algorithm or somewhere in the imglib2 domain (this is up to @tpietzsch :-)). I think extending Shapes to work with arbitrary StructuringElements or better implementing a Shape which can do that is a good approach.

In imagej-ops I would put the actual implementations of morphological ops, based on Shapes/Neighborhoods. This means offering interfaces and implementations for Erode / Dilate / Open / Close etc. based on the Shapes in imglib2, such that someone can call for exampleops.run("erode", image, stucturingElement). If those implementations make use of other imagej-ops (for example MapNeighborhood) then they directly benefit from all the multi-threading and optimization taking place underneath the hood and you don't have to care about multi-threading at all. Additionally, if you use the imagej-ops class hierarchy, you also can apply your algorithm on arbitrary dimension combinations in your input images (e.g. run the morphological op plane-wise in a video etc. see: HyperSlice and the corresponding JUnits tests for examples).

But to answer your question: Yes. ImageJ-Ops can also just be wrappers for imglib2-algorithms (e.g. as we do it with Gauss).

[tinevez]

Ok @dietzc, There will be high-level calls both for ops and imglib2, that both call the imglib2 strels, but are independent. Ok?

[dietzc]

Sure, sounds like a plan. I think as soon as the Shape can take arbitrary StructuringElements, the rest will be straight-foward. So I agree, most of the effort has to be spend on imglib2 side I guess. Because once you have that, you can write simple Ops (e.g. erode, dilate) which are then consumed by the Map op.

Let me know if I can support you somehow!

[iarganda]

Thanks @ctrueden!

[tinevez]

Hi all,

WIP:

• Current WIP is documented here: http://fiji.sc/Imglib2_Morphological_Operations
• Core classes for Imglib2 are here: https://github.com/tinevez/imglib2-algorithm/tree/morphology/src/main/java/net/imglib2/algorithm/morphology
• Interactive tests for the framework are here: https://github.com/tinevez/imglib2-tests/tree/morphology/src/test/java/net/imglib2/algorithm/morphology

What is missing:

• The ImageJ-Ops implementation.
• Still depends on deprecated classes (Multithreading related)
• Please tell me.

What is not here:

• True gray morphology: morpho. ops. on gray images with gray strels. Like what is needed to do true rolling-ball background subtraction. This can do morpho ops on gray images with B&W strels.

Opinions as what should be done next is welcome. JY

[dietzc]

Hi @tinevez, this looks great. I just have some minor suggestions/questions:

• As Curtis/Harris pointed out, you could use an OutOfBoundsStrategy on the Source image instead of testing if the neighborhood is still contained in the source interval (e.g.https://github.com/tinevez/imglib2-algorithm/blob/morphology/src/main/java/net/imglib2/algorithm/morphology/MorphologicalOperations.java#L284). This would be a more flexible solution, because the user can decide what happens with the pixels on the border.
• For binary images, one could offer optimized erode/dilate implementations which stop searchig for min/max if they already found 0 or 1. Maybe thats faster, I don't know.
• Question: Everything you did in https://github.com/tinevez/imglib2-algorithm/blob/morphology/src/main/java/net/imglib2/algorithm/morphology/MorphologicalOperations.java will be somehow done in Ops using the Ops infrastructure right? If you want, I can do that on Ops side and you comment on it? Like that you would get used to Ops and you don't have to do everything on your own.
• Question: The Neighborhood implementations are really cool. Will that stuff go into imglib2-roi? @tpietzsch what do you think?

Again, thats great stuff. I'm really looking forward to see it in ops/imagej2/knime/fiji etc :-)

[tinevez]

Thanks @dietzc

1. Ok for the OutOfBounds strategy. Anyway I have to rewrite the MorphologicalOperations class and split it in individual operations. Right now, the methods are not imglibiesque enough. As @tpietzsch, @StephanPreibisch and @Squareys advised, there should also be methods that take a RandomAccessible as source and write results in a provided IterableInterval.
2. Binary image optimization: great idea.
3. ImageJ ops version: Ok!
4. The neighborhood is @tpietzsch design, after I think several iterations. I do not know what's his plan for imglib2-roi.

Thank again!

[ctrueden]

I just browsed the Imglib2 Morphological Operations wiki page, and holy cow has a lot of work gone into this over the past two years. And it's still not merged to master anywhere? What are the obstacles there? Is there anything a student at LOCI or Uni Konstanz could do to help?

[dietzc]

@tibuch will work on this after he finishes the view stuff. Should be ready end of August.

[ctrueden]

@landinig wrote:

you could consider that the border of the image is the end of the universe (and it gets eroded, as outside it is 0), or you can asume that this is a window in a much larger universe (and so you assume it is filled) and remains not eroded. Some might not worry too much about this detail, but there are good reasons for dealing with those distinct 2 cases, it would break a large amount of our code if that is changed.

Just wanted to comment that both of those cases are fully covered in ImgLib2 by the use of "out of bounds strategies". So I think there is no problem here. There is even an out of bounds strategy which extends the edge according to the border values, which might be nice in some cases here too.

[ctrueden]

Addressed in #358.