Add `image` Type of Trace

[RoyiAvital]

For MATLAB, Python and JS there is a support for the image type of trace.

It would be great to have it in Julia as well.

By the way, please make it support AbstractArrays and not only the image type from Julia Images.

By the way, there is some reference for Julia as well, but not in the official documentation.

[empet]

There is full support for image trace. Please take a look at these notebooks: https://github.com/empet/PlotlyJS-image-tutorials.

[RoyiAvital]

@empet , I guess what I was after is something similar to Python or MATLAB. Namely being able to display a Matrix{T} with color mapping (Grayscale for single channels images, colormap if it is not real image) and Arrray{T, 3}.

Personally, I'd be happy to stay away from Matrix{RGB{T} and Matrix{RGBA{T}. I prefer not having Images.jl as a dependency.

[empet]

It's surprising that you need to generate an image from an Array{T, 3}, as long as Julia images are either Matrix{RGB{T}} or Matrix{RGBA{T}}. go.Image from plotly.py, as well as plotly.express.imshow can display arrays of dtype =np.uint8. As a consequence PlotlyJS.image can display only an Array{UInt8, 3}. You can find a few examples in a new notebook added here https://github.com/empet/PlotlyJS-image-tutorials/tree/main, as https://nbviewer.org/github/empet/PlotlyJS-image-tutorials/blob/main/1p-Image-from-ArrayUInt8.ipynb. All files and images involved in this repo are available in the folder images. Let us know if there are more cases you've seen in plotly.py or Matlab version of plotly .

[RoyiAvital]

The reason is for performance it is better to use planar from for image. The design choice of Julia Images was to use Array of Structs, but in other languages images are usually, for performance, struct of arrays. Hence the need to support Matrix{T} for single channel images and Array{T, 3} for multi channel images.

I guess being able to replicate what's on the Python page will be great: https://plotly.com/python/imshow/. It doesn't seem to be limited to UInt8 in those examples. They seem work on integers and floating point.

[empet]

Running each example from imshow tutorials, https://plotly.com/python/imshow/), and inspecting fig.data you realize whether the image is defined from source or from an array (as well as its type). Only if px.imshow creates a heatmap, the matrix can have a dtypedifferent of np.uint8.

[RoyiAvital]

I am not sure what you mean, but the documentation says they support input which is floating point:

[empet]

To convince yourself just run:

Plot(image(z=permutedims(rand( 3, 10, 7), (1, 3,2)), colormodel="rgb")) #colormodel="rgb"   is the default colormodel

and

Plot(image(z=permutedims(rand(0:255, (3, 10, 7)), (1, 3,2)), colormodel="rgb")) 

In the first Plot the typeof(z) is Array{Float64, 3}, while in the second one Array{Int64, 3}, which is interpreted correctly by plotly.js.

LE: Even

Plot(image(z=permutedims(rand(100:300, (3, 10, 7)), (1, 3,2)), colormodel="rgb")) 

works, and this means that somewhere behind the scenes the Int64 values are converted to UInt8.

[RoyiAvital]

I ended up with:

function DisplayImage(mI :: Matrix{T}; tuImgSize :: Tuple{N, N} = size(mI), titleStr :: String = "" ) where {T, N <: Integer}
    # Displays a grayscale image in the range [0, 1]

    oTr1 = heatmap(z = UInt8.(round.(255 * mI))[end:-1:1, :], showscale = false, colorscale = "Greys");
    oLayout = Layout(title = titleStr, width = tuImgSize[2] + 100, height = tuImgSize[1] + 100, 
                hovermode = "closest", margin = attr(l = 50, r = 50, b = 50, t = 50, pad = 0));

    hP = plot([oTr1], oLayout);

    return hP; #<! display(hP);

end

function DisplayImage(mI :: Array{T, 3}; tuImgSize :: Tuple{N, N} = size(mI)[1:2], titleStr :: String = "" ) where {T, N <: Integer}
    # Displays an RGB image in the range [0, 1]

    oTr1 = image(z = permutedims(UInt8.(round.(255 * mI)), (3, 2, 1)), colormodel = "rgb");
    oLayout = Layout(title = titleStr, width = tuImgSize[2] + 100, height = tuImgSize[1] + 100, 
                hovermode = "closest", margin = attr(l = 50, r = 50, b = 50, t = 50, pad = 0));

    hP = plot([oTr1], oLayout);

    return hP; #<! display(hP);

end

The nice thing, it also allows 1:1 display of the image (Each pixel of the image is displayed by a single pixel on the screen).

I find PlotlyJS.jl to be the best plotting library in Julia. Thank you for your work!