Converters with LabelingTypes

[dietzc]

@tpietzsch : I have a general question concerning the labeling implementations in https://github.com/imglib/imglib2-roi/tree/revised-labeling.

Assume I have a RandomAccessibleInterval<ShortType> as input. I want to have a RandomAccessibleInterval<LabelingType<ShortType>> as output. The value at each position in input should be the 'label' of the output. It's obvious that a Converter<ShortType, LabelingType<ShortType> would make sense. However, there are some problems. For example, I have to create a LabelingType<ShortType> which I can pass to the Converter. I can't pass null for the LabelingMapping of the desired LabelingType as getMapping() on the LabelingType should return a valid LabelingMapping. I thought about implementing an own LabelingMapping for this particular use-case, but then I'm not sure how to implement getLabels() or labelsAtIndex(int idx).

I was thinking about the minimal interface for a LabelingMapping (e.g. a Converter from IntegerTypeand some Set<L> + Externalization methods inside the mapping,). But I have no clue if this makes sense at all.

Do you have any ideas how I could nicely use Converter in the case of Labelings?

[tpietzsch]

@dietzc I fear that there is no perfect solution for this. One way to go is the one you outlined: make your own LabelingMapping. We would have to open up the LabelingMapping API a bit, that is make some methods that are now package private into protected methods such that they can be overridden. The problem is that without looking at all pixels of the image you want to convert you have no way of knowing which "labels" are defined. That means, getLabels() would have to return the set of all possible values of the index type, ie, {0, ..., Integer.MAX_VALUE} for IntType. labelsAtIndex(idx) would simply return the set {idx}. Problem is that LabelRegions would become very memory intensive because it creates a properties aggregator for each label set (eg, Integer.MAX_VALUE many for IntType). It is just not a good fit for this problem.

Second option would be to have a more generic LabelingType, for example

public interface GenericLabelingType<G extends GenericLabelingType<G, T>, T>
        extends Type<G>, Set<T>
{}

which would be implemented by the current LabelingType. However, stuff that runs on LabelingType such as LabelRegions would not run on this GenericLabelingType. They need to know that about LabelingMapping to be efficient. We could of course make a (less efficient) GenericLabelRegions. I could try to do that, but what would you need it for specifically? Maybe there is a better solution.

[dietzc]

I already thought that it wouldn't be possible. So here are some use-cases where we used converters:

• Simply append a prefix to each label in a Labeling which was e.g. created by a connected component analysis. This was frequently used if two Labelings are merged in a subsequent step (e.g., to distinguish between nuclei and chromosomes (or so)).
• Convert an Img<IntegerType> to a Labeling. Often, one can export labelings as integer images from 3rd party software. We used a converter to represent those images as Labelings (which was not correct, as the mapping was wrong). But in this case we really should actually convert the Img to a Labeling by iterating over the Img and setting the value in the Labeling (to determine domain of getLabels() etc).
• Renaming of labels: We have a UI in which we enable users to rename certain labels by clicking on one component in the Labeling. In this case we used a Converter to present an updated Labeling (i.e. with the renaming) to our renderer (which works on label basis) without recalculating all regions. If there would be an efficient incremental update of derived LabelRegions of a Labeling, I think we could avoid Converters in this use-case (I'm afraid this wouldn't work, as the RandomAccessibleInterval<LabelingType<L>> doesn't know about any derived LabelRegions. Right?

The more I think about it, the more I understand that in many of our use-cases we only need read-only access to a converted label. Also, I like the to introduce a more general interface for LabelingMapping. However, I fear that we just move the computation time which is lost by e.g. converting a Img<ShortType> to an ImgLabeling<String, ShortType> to method calls on the LabelingMapping, e.g. when calling getLabels() we need to actually parse the Img.

[tpietzsch]

@dietzc I thought about it some more. In principle you should be able to do what you would like to do using the LabelingMapping.SerialisationAccess (http://github.com/imglib/imglib2-roi/blob/revised-labeling/src/main/java/net/imglib2/roi/labeling/LabelingMapping.java#L323) to modify the mapping of the "converted type". It would not be too difficult to add something where you can do this more fine-grained, i.e., modify the sets associated with individual indices. However, this will only work correctly if

• nothing is ever added to the empty set (index 0 always)
• no changes are made that would lead to structural modifications. That is, if you have pixels labeled with {A,B} and pixels labeled with {B,C} then changing both sets to be {A,B,C} will not make the pixels labeled so have the same index and things will probably start going horribly wrong.

I think it is just a bad idea to do this.

There should be better ways to do the things you mentioned. Perhaps you could think about making the Labels themselves a bit more powerful. For example in the UI example, a label does not have to be an Integer. It could be an object that has an Integer and a boolean selected flag that is used by your renderer etc. Modifications to the label objects (e.g. setting the selected flag) will not trigger any changes in the Labeling, LabelingRegions, etc. Would that work?

[tpietzsch]

@dietzc I made a small example of what a more generic LabelingType (it doesn't even have a LabelingMapping) could look like. It is in the branch https://github.com/imglib/imglib2-roi/tree/revised-labeling-converters. Could you have a look and provide some feedback if that would be a useful direction for you? Note that as I said earlier LabelRegions etc will not (and cannot) work with arbitrary GenericLabelingType, but maybe you don't use that anyway in these cases???

[imagejan]

Sorry to revive this old issue, but just for the record:

@dietzc wrote:

But in this case we really should actually convert the Img to a Labeling by iterating over the Img and setting the value in the Labeling (to determine domain of getLabels() etc).

I now submitted https://github.com/imagej/imagej-common/pull/93 which adds a new org.scijava.convert.Converter to convert from Img to net.imglib2.roi.labeling.ImgLabeling. It was loosely inspired by the way how this is currently done in KNIME, but I think it would be good to have this functionality available upstream in ImageJ as well.

[ctrueden]

I released imagej-common 0.33.0 containing @imagejan's new converters.