Format of canvas image_data

[deepika2502]

To further proceed with the canvas result, image_data, I need to know its type to work with opencv. Can you suggest a way that will work for opencv functions as well as displaying the image with st.image()

[andfanilo]

Hello @deepika2502 ,

Thanks for reporting, you're right I didn't specify the typing of image_data in the docs. It's a (height, width, 4) numpy array representing the RGBA values at the given pixel position. You can directly display if with st.image(canvas_result.image_data).

I did not test but judging from https://stackoverflow.com/questions/7587490/converting-numpy-array-to-opencv-array#7587819 you'll need to convert the numpy array to cv array beforehand with cv.fromarray.

You can also convert if to a Pillow image, which I think OpenCV can work on:

from Pillow import Image
im = Image.fromarray(img_data)

Hope this helps, Fanilo

[deepika2502]

Hello, Thank you for the response. I figured it last night and converted the canvas image to opencv format, eliminating the A from RGBA. I cannot display the image using st.image() now. It gives a black picture

[deepika2502]

Hello, Thank you for the response. I figured it last night and converted the canvas image to opencv format, eliminating the A from RGBA. I cannot display the image using st.image() now. It gives a black picture

Hey, I figured it out. Thank you (:

[komodovaran]

For anyone else who stumbles across this and wonders "what was the answer in the end", I've made it work with the following, assuming img_2d is a 2D greyscale image (numpy array) with float intensities in the range (0, 1):

from PIL import Image # note that this is using 'Pillow'

display_img = (
    Image.fromarray((img_2d * 255).astype(np.uint8))
    .resize(img_2d.shape)
    .convert("RGB")
    )

canvas_result = st_canvas(..., background_image=display_img)

[deepika2502]

For anyone else who stumbles across this and wonders "what was the answer in the end", I've made it work with the following, assuming img_2d is a 2D greyscale image (numpy array) with float intensities in the range (0, 1):

from PIL import Image # note that this is using 'Pillow'

display_img = (
    Image.fromarray((img_2d * 255).astype(np.uint8))
    .resize(img_2d.shape)
    .convert("RGB")
    )

canvas_result = st_canvas(..., background_image=display_img)

Hello @komodovaran, thank you for adding a solution

[murtazabasu]

Sorry, if I didn't understand this correctly. But in my case after uploading an image and drawing something on it I would like to get the drawing as well as the image that was uploaded back. st.image(canvas_result.image_data) only displays the drawing and not the uploaded image in bg.

[andfanilo]

Hi @murtazabasu,

Unfortunately it is not possible for now :/ it's technically a bit of a pain to do as the drawing and background image are on different canvases on the JS side. I think you should be able to superimpose the drawing and the image on the Python side though (https://pythonexamples.org/python-opencv-add-blend-two-images/)

[murtazabasu]

Sorry, if I didn't understand this correctly. But in my case after uploading an image and drawing something on it I would like to get the drawing as well as the image that was uploaded back. st.image(canvas_result.image_data) only displays the drawing and not the uploaded image in bg.

Thank you for the reply @andfanilo I solved it using PIL.

bg_image = st.sidebar.file_uploader("Base image:", type=["png", "jpg"]) 
image = Image.open(bg_image)
canvas_result = st_canvas(  
    stroke_width=stroke_width,
    stroke_color=stroke_color,
    background_image=image,
    update_streamlit=True,
    height=image.size[1],
    width=image.size[0],
    drawing_mode=drawing_mode,
    key=key
)
uploaded_image = Image.fromarray(np.array(image))
drawing = Image.fromarray((canvas_result.image_data).astype(np.uint8))
uploaded_image.paste(drawing, (0,0), mask=drawing)  # super imposed image