mlverse / torchvision

R interface to torchvision
https://torchvision.mlverse.org
Other
63 stars 15 forks source link

Feature/utils draw bbox #80

Closed cregouby closed 2 years ago

cregouby commented 2 years ago
library(torchvision)

image <- (255 - (torch::torch_randint(low = 1, high = 60, size = c(3, 360, 360))))$to(torch::torch_uint8())
x <- torch::torch_randint(low = 1, high = 160, size = c(12,1))
y <- torch::torch_randint(low = 1, high = 160, size = c(12,1))
dog_boxes <- torch::torch_cat(c(x, y, x + runif(1, 40, 60), y +  runif(1, 50, 100)), dim = 2)
bboxed_image <- draw_bounding_boxes(image, dog_boxes, labels = "dog")

tensor_image_display(bboxed_image)

Created on 2022-09-13 by the reprex package (v2.0.1)

gives a X11 window with image

and

library(torchvision)

image <- (255 - (torch::torch_randint(low = 1, high = 60, size = c(3, 360, 360))))$to(torch::torch_uint8())
x <- torch::torch_randint(low = 1, high = 160, size = c(12,1))
y <- torch::torch_randint(low = 1, high = 160, size = c(12,1))
word_mask <- torch::torch_cat(c(x, y, x + runif(1, 40, 60), y +  runif(1, 5, 7)), dim = 2)
bboxed_image <- draw_bounding_boxes(image, word_mask, color = "black", fill = TRUE)

tensor_image_display(bboxed_image)

Created on 2022-09-13 by the reprex package (v2.0.1)

giving image

cregouby commented 2 years ago

Hello @dfalbel, I may need some help to pass the R-CMD-check / ubuntu-20-04 (release) workflow. It seems the machine is missing a font

  >   bboxed <- draw_bounding_boxes(image, boxes, colors = "black", fill = TRUE)
  Error in graphics::mtext(sprintf("help(\"%s\")", nameEx()), side = 4,  : 
    R: unable to read font `helvetica' @ error/annotate.c/RenderFreetype/1338
  free(): double free detected in tcache 2
  Aborted (core dumped)

and it seems linked to this https://github.com/ropensci/magick/issues/174

I don't know if it is a good idea to bypass the test (what I'm trying to do right now) or if there is a way to add the font in the ubuntu-20-04 image in the CICD pipeline (for which I've no idea on how to).

dfalbel commented 2 years ago

Sorry for taking so long to work on this PR @cregouby . I have made a few changes to the code. Mainly:

  1. Renamed the display and browse functions,
  2. Removed a $tile() in the draw_segmentation_mask which I believe were an error.
  3. Modified CI scripts.

Do you want to take a look?

FWIW the Windows errors seem to be spurious. I think they will solve automatically in a few days.

cregouby commented 2 years ago

Great for me ! Thanks for the rework !

dfalbel commented 2 years ago

Thank you! Merging now!