Evision.Mat.update_roi crashing BEAM

[gworkman]

Hi! Love this project, and hoping I can start to contribute sometime soon :)

I am having some trouble using Evision.Mat.update_roi, and it results in crashing the BEAM (or at least my livebook runtime). I haven't figured out the correct parameters to pass in order to 'paste' one image on top of another. Similar numpy code would look like the following:

def draw_image(image, image_to_draw, x, y):
    h, w = image_to_draw.shape[:2]
    image[y:y + h, x:x + w, :] = image_to_draw
    return image

Now, when I try to do the same with Evision, my implementation is below:

  def draw_image(image, image_to_draw, x, y) do
    {h, w, _} = image_to_draw.shape
    roi = [{y, y+h}, {x, x+w}, {0, 2}]
    Evision.Mat.update_roi(image, roi, image_to_draw)
  end

This results in the Livebook runtime crashing and the terminal printing the following error:

libc++abi: terminating with uncaught exception of type cv::Exception: OpenCV(4.6.0) /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/core/src/matrix.cpp:879: error: (-215:Assertion failed) (int)ranges.size() == d in function 'Mat

It looks like I'm doing something wrong with my indexing? Or just the general format of the roi parameter.

I think the docs could use some clarification or examples on what the correct roi parameter looks like. Example: are the {start_index, end_index} of the ROI inclusive? (I'd be happy to contribute after getting some clarity on these points!). And also it would of course probably be good to catch the error as a {:error, "invalid ROI"} tuple :)

Let me know what you think.

Thanks and happy new year!

Gus

EDIT:

It does not crash when I use the following implementation (which removes the third channel from the ROI), but it also doesn't update the ROI as I would expect:

  def draw_image(image, image_to_draw, x, y) do
    {h, w, _} = image_to_draw.shape
    roi = [{y, y+h}, {x, x+w}]
    Evision.Mat.update_roi(image, roi, image_to_draw)
  end

[gworkman]

Alternative implementation that is working via Nx, in case anyone else has this issue in the meantime:

  def draw_image(image, image_to_draw, x, y) do
    image_nx = Evision.Mat.to_nx(image, EXLA.Backend)
    image_to_draw_nx = Evision.Mat.to_nx(image_to_draw, EXLA.Backend)

    new_image = Nx.put_slice(image_nx, [y, x, 0], image_to_draw_nx)
    Evision.Mat.from_nx_2d(new_image)
  end

[cocoa-xu]

Hi @gworkman, thanks for reporting this bug! I rewrote this function in C++, and it should work now. :)

[cocoa-xu]

I plan to release v0.1.26 later today. The reason for the delay is that I saw that all unit tests related to the dnn module constantly crash BEAM on macOS CI (macOS only, all the unit tests passed on Linux/Windows). However, I cannot reproduce the crash on my M1 and Intel Macs.

I'll run the same tests in a virtualized macOS, and if I can reproduce the bug in the virtual machine, I'll solve the issue first and then release v0.1.26. Otherwise, I'll probably release it and see if any Intel Mac users report this issue.

[gworkman]

Big thank you for your work on this! I am an intel Mac user and can run some tests if needed