DecodeNumpy via BufferAccess classes.

[mkitti]

Here's an alternate version of the DecodeNumpy.build.

https://github.com/bioimage-io/model-runner-java/blob/ea4bec4616d81ce63c3fb1ed4e6c13aeb0e4c53c/src/main/java/io/bioimage/modelrunner/numpy/DecodeNumpy.java#L248

This may require a version of ImgLib2 with https://github.com/imglib/imglib2/pull/299 . The earliest such version would be imglib2-5.13.0.

Currently, pom-scijava is at imglib2-5.12.0 but @ctrueden mentioned he was interested in release a pom-scijava with a version bump to imglib2-5.13.0.

The main reason for this change is so that it does not copy the ByteBuffer but rather uses it directly.

public static <T extends NativeType<T>> Img<T> build(ByteBuffer buf, ByteOrder byteOrder, String dtype, long[] shape) throws IllegalArgumentException
    {
        buf.order(byteOrder);
        if (dtype.equals("byte")) {
            ByteAccess access = new ByteBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.bytes( access, shape );
        } else if (dtype.equals("ubyte")) {
            ByteAccess access = new ByteBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.unsignedBytes( access, shape );
        } else if (dtype.equals("int16")) {
            ShortAccess access = new ShortBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.shorts( access, shape );
        } else if (dtype.equals("uint16")) {
            ShortAccess access = new ShortBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.unsignedShorts( access, shape );
        } else if (dtype.equals("int32")) {
            IntAccess access = new IntBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.ints( access, shape );
        } else if (dtype.equals("uint32")) {
            IntAccess access = new IntBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.unsignedInts( access, shape );
        } else if (dtype.equals("int64")) {
            LongAccess access = new LongBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.longs( access, shape );
        } else if (dtype.equals("float32")) {
            FloatAccess access = new FloatBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.floats( access, shape );
        } else if (dtype.equals("float64")) {
            DoubleAccess access = new DoubleBufferAccess(buf, true);
            return (Img<T>) ArrayImgs.doubles( access, shape );
        } else {
            throw new IllegalArgumentException("Unsupported tensor type: " + dtype);
        }
    }

[mkitti]

xref: https://github.com/scijava/pom-scijava/issues/229

[carlosuc3m]

Awesome, I will implement it then. Is it possible to get a reference to the imglib2 buffer too? Does the cursor.get().getByte() already return a reference?

[carlosuc3m]

Hello @mkitti , I am trying to use your code and I am encountering a couple of problems. When trying to create imglib2 images from npy files that contain 3D images the reconstruction doe snot work well. I think it is because ImgLib2 indexes flat arrays in a different way to numpy. This is why with 2D images it works well.

Is there a way to go around this problem using ImgLib2 code?

[mkitti]

I am uncertain what you mean by doss not work well.

My guess is that you need to reorder or permute the dimensions. Numpy is usually row major, and I believe imglib2 is column major.

Numpy does support Fortran order.

https://numpy.org/doc/stable/reference/generated/numpy.asfortranarray.html

You could try that. Otherwise, you will need to permite the dimensions on the ImgLib2 side.

[carlosuc3m]

This is what I was trying to refer to:

My guess is that you need to reorder or permute the dimensions. Numpy is usually row major, and I believe imglib2 is column major.

But I did not know how ImgLib2 dimensions are organised.

Permuting the axes might be the best option. Thanks a lot @mkitti !

[mkitti]

Numpy has dimensions (y,x) where the last dimension is the fastest varying dimension. ImgLib2 has dimensions (x,y) where the first dimension is the fastest varying dimension.

For 3D, Numpy would be (z,y,x) and ImgLib2 would be (x, y, z).

[ctrueden]

Meanwhile, pom-scijava 37.0.0 has been released and now manages imglib2 at 6.2.0...

[carlosuc3m]

Finally implemented for numpy file reading. Also improved the tensor<->imglib2 RAI conversion accross all the Deep Learning engines with the help of @tpietzsch at Brno's Hackaton Oct 2023 https://github.com/bioimage-io/JDLL/blob/ec14b9db02e3703f98e1da7775310b766b4c3f30/src/main/java/io/bioimage/modelrunner/numpy/DecodeNumpy.java#L325