"KNIP"-Projectors (sampling and dimension selection)

[MichaelZinsmaier]

Hi all,

I work togehter with Christian and Martin on the KNIME Image Processing Plugin. We wanted to use something similar to the net.imglib2.display.XYProjector to render images for our views, however the current implementation was a bit too restricted for our use cases.

Therefore we implemented a few new classes that might be interesting for you as well. The implementation can be found at net.imglib2.display.projectors in branch knip or projectors.

We use these projctors since august last year as basis for the Knime Image Processing Image Views and I thought posting it here would be a good way to make the classes visible for interested developers.

a short description from an earlier e-mail

The code is organized as follows:

• Projector2D: is similar to the XYProjector but it supports also the selection of other dimensions (the code is not active because it requires the class Subsetviews from ops, that is not part of the projectors branch)
• DimProjector2D: is similar to the CompositeProjector however instead of providing multiple Converters the idea is to provide a single converter with access to multiple values from the "projected" third dimension. This gives the converter more flexibility with respect to the composition of single conversion results.
• specializedProjectors: contains some 2DProjectors that allow fast (normalized) extraction of 2D greyscale datasets from certain storage strategies (Planar/Array)
• screenimages: contains classes that convert Byte/Short ArrayImages into Java Images similar to the implementation of net.imglib2.display.ARGBScreenImage

I'm looking forward for your comments

Regards Michael

[ctrueden]

I think these additions are very useful!

Note that for such code additions, GitHub has a great Pull Request (PR) feature, which requests that your branch be merged into the mainline upstream. Rather than only mentioning branch names in the issue description, it makes it very easy for everyone to code review since the commits are listed directly in the thread. And the PR stays automatically up to date with the state of the branch, via additional commits or even force push as needed.

Looking at the projectors branch, I see that there are several new packages: net.imglib2.display.projectors and subpackages. What does the core ImgLib2 team (@StephanPreibisch, @axtimwalde, @tpietzsch) think about the naming?

I like the idea of the specialized projectors, but I wonder: is there any factory or heuristic code for deciding whether a given projector specialization can be used? Or must it be done manually, with a priori knowledge? If there were a way to automatically create the most appropriate and performant projector from a given input Img, that would be awesome.

[MichaelZinsmaier]

A bit more information about the specialized projectors:

Currently we implemented 4 specialized projectors. These projectors allow us to copy data from a Byte/Short planar/array img directly (System.arraycopy) into the storage of a Byte/Short ScreenImage which uses the data as basis for its BufferedImage member. (optionally the values can be normalized after copying)

This allows very fast rendering of standard cases like XY plane of an ArrayImg rendered as grayscale.

But we have to test the images to decide if the projectors can be applied:

        // try ArrayImage
        tryArrayImage(source, dimX, dimY, planePos, normalizationFactor, min);

        // try PlanarImage
        tryPlanarImage(source, dimX, dimY, planePos, normalizationFactor, min);

the tests look like this:

        if (2d rendering)
                if ((dimX == 0) && (dimY == 1) && (source instanceof ArrayImg)) {
                       if ((type == NativeTypes.BYTETYPE) || (type == NativeTypes.UNSIGNEDBYTETYPE)) {

                            //apply ArrayImg byte specialization

[ctrueden]

Thanks @MichaelZinsmaier. Like I said, I really like these changes, but I would prefer that at least one of the three main authors (@tpietzsch, @axtimwalde, @StephanPreibisch) sign off on it before merging.

I also think it would be cool to somehow select the most performant projector automatically (the SciJava Common plugin framework is perfect for making this sort of thing fully extensible), but do not have time to work on it myself, since we do not urgently need it in ImageJ2. (Eventually, we'll need it for performance reasons though, I expect.) So if you want to take a stab at automating the projector auto-selection, and would like some more information on how to do it with a SciJava service, let me know and I can expound.

[tpietzsch]

Hi,

I think that this stuff is useful but it needs some work. ByteScreenImage and ShortScreenImage I find very useful we should have those definitely. Also the specialised projectors are useful but it would be nice if the existing and new projectors could be unified into a single class hierarchy. Probably the XYProjectors should be Abstract2DProjectors. Also CompositeXYProjector and DimProjector2D are very related. Both are also related to Saalfeld's recently committed Composite and CompositeView (which would benefit from some javadoc btw.) Stephan, what do you think? Could/should this be unified?

Besides this "big-picture" issues, here are some more suggestions/questions:

• Abstract2DProjector should extend Point instead of reimplementing Positionable and Localizable, thereby saving a lot of code (see AbstractXYProjector).
• Please re-format to imglib2 code style (code stye for eclipse is in core/doc/imglib2-eclipse-code-style.xml)
• Can you explain better what the normalization is supposed to do? More javadoc would be helpful. Looking at the code it seems that this is quite specific to the unsigned types, however for instance ArrayImgXYByteProjector is supposedly working with GenericByteType? There also is the isSigned parameter in the constructor which somehow is related to this. This just seems not a clean solution. isSigned can be inferred from the type. Maybe we should just add more specialised classes?
• ProjectedDimSamplerImpl is an interface? Thats confusing. I think we shouldn't name an interface *Impl.
• Why is the completely commented-out Projector2D there?

The following is not form the projectors branch, but while we are at it:

• UnsignedByteScreenImage should extend ArrayImg like the other ScreenImages
• the Converter implementations are not specific for display. So they could go into a subpackage of net.imglib2.converter. Any suggestions? net.imglib2.converter.converters?
• Volatile, VolatileNumericType, VolatileRealType should not be in package display.

best regards, Tobias

On Jun 28, 2013, at 8:57 PM, Curtis Rueden notifications@github.com wrote:

Thanks @MichaelZinsmaier. Like I said, I really like these changes, but I would prefer that at least one of the three main authors (@tpietzsch, @axtimwalde, @StephanPreibisch) sign off on it before merging.

I also think it would be cool to somehow select the most performant projector automatically (the SciJava Common plugin framework is perfect for making this sort of thing fully extensible), but do not have time to work on it myself, since we do not urgently need it in ImageJ2. (Eventually, we'll need it for performance reasons though, I expect.) So if you want to take a stab at automating the projector auto-selection, and would like some more information on how to do it with a SciJava service, let me know and I can expound.

— Reply to this email directly or view it on GitHub.

[MichaelZinsmaier]

Hi,

I think the XYProjector could be implemented as a special case of the Projector2D (see below).

~~~~  small picture issues ~~~~

=> Abstract2DProjector now extends Point
=> reformatted using the checkstyle
- Can you explain better what the normalization is supposed to do? More javadoc would be helpful. Looking at the code it seems that this is quite specific to the unsigned types, however for instance ArrayImgXYByteProjector is supposedly working with GenericByteType? There also is the isSigned parameter in the constructor which somehow is related to this. This just seems not a clean solution. isSigned can be inferred from the type. Maybe we should just add more specialized classes?

  => signed parameter removed
  => I added some comments to the normalization especially in the ArrayImg Byte version
           basically it is
           factor = (typeMax - typeMin) / (newMax - newMin)
           min = newMin
           normalizedValue = (value - min) \* factor

=> ProjectedDimSamplerImpl renamed to ProjectedDimSamplerRandomAccess
    it is just a subinterface of DimSamplers that requires RandomAccess for its sampling strategy

=> Projector2D is intended as replacement for the XYProjector. Like the XYProjector it applies a converter to a 2D slice and stores the result in a 2D target. But it allows to specify the slice dimensions and is not limited to X=0, Y=1. This is achieved using SubsetViews that are not part of the projectors branch.  Therefore it is not active at the moment.
One more difference, I don’t think the Projector2D would work on 1D sources and targets right now.
But this could be added if necessary.

best regards,
Michael

[ctrueden]

Sounds like good progress. Could you please submit your latest version of the code as a Pull Request to the imglib codebase? It will be easier for @tpietzsch and others to review the commits, then. Then we can close this issue with a reference to the PR (which is itself also an issue...).

To create a PR, you first push a topic branch to the imglib codebase, then browse it on GitHub using the "Compare & Review" button (green arrows to the left of the branch selector). Then click the box which says "Click to create a pull request for this comparison".

Thanks!

[MichaelZinsmaier]

opended a pull request #23 closed this issue