as.cimg.magick-image

[jwijffels]

Would it make sense to add such a function to the imager package? Or is there a quicker alternative for the conversion?

library(imager)
library(magick)
library(abind)
"as.cimg.magick-image" <- function(x){
  out <- lapply(x, FUN=function(frame){
    frame <- as.integer(frame[[1]])[, , 1:3] # dropping the alpha channel
    dim(frame) <- append(dim(frame), 1, after = 2)
    frame
  })
  out$along <- 3
  out <- do.call(abind::abind, out)
  out <- as.cimg(out)
  out <- permute_axes(out, "yxzc")
  out
}
banana <- image_read("https://jeroenooms.github.io/images/banana.gif")
x <- as.cimg(banana)
plot(x, 7)

[dahtah]

Thanks for your contribution! I've taken the liberty of rewriting your code to remove the dependency on abind. I also keep the alpha channel for users to do as they wish. Are you happy with the following?

cvt.frame <- function(f)
{
    f <- as.double(f)
    d <- dim(f)
    dim(f) <- c(d[1:2],1,4)
    cimg(f) %>% imrotate(90) %>% mirror("x")
}

"as.cimg.magick-image" <- function(im)
{
    map_il(seq_len(length(im)), ~ cvt.frame(im[[.]])) %>% imappend("z") 
}

[jwijffels]

This looks fine for me, although I haven't looked into detail to the structure of the magick-image. Is it always an array with 4? It looks like it does at https://github.com/ropensci/magick/blob/master/R/base.R at "[[.magick-image" but maybe a unit test needs to be added to avoid the dimension will be changed later on.

Reason why I dropped the alpha channel in the code was that grayscale did not work as show below.

cvt.frame <- function(f, channels)
{
  f <- as.double(f)
  if(!missing(channels)){
    f <- f[, , channels]
  }
  d <- dim(f)
  dim(f) <- c(d[1:2],1,length(channels))
  cimg(f) %>% imrotate(90) %>% mirror("x")
}

"as.cimg.magick-image" <- function(im, channels = 1:3)
{
  map_il(seq_len(length(im)), ~ cvt.frame(im[[.]], channels = channels)) %>% imappend("z") 
}

frink <- image_read("https://jeroenooms.github.io/images/frink.png")
x <- as.cimg(frink)
grayscale(x)

x <- as.cimg(frink, channels = 1:4)
grayscale(x)
Error in grayscale(as.cimg(frink)) : 
  [instance(220,445,1,4,0000000014fca040,non-shared)] CImg<double>::RGBtoHSL(): Instance is not a RGB image.

Other thing I haven't looked at in detail is what happens when the width/height differ by frame in the magick package. Looks like imappend is doing that, with abind you get the error message of inconsistent dimensions

bigdata <- image_read('https://jeroenooms.github.io/images/bigdata.jpg')
frink <- image_read("https://jeroenooms.github.io/images/frink.png")
logo <- image_read("https://www.r-project.org/logo/Rlogo.png")
img <- c(bigdata, logo, frink)
img <- image_scale(img, "300x300")
image_info(img)
  format width height colorspace filesize
1   JPEG   300    225       sRGB        0
2    PNG   300    263       sRGB        0
3    PNG   148    300       sRGB        0
x <- as.cimg(img)
dim(x)
[1] 300 300   3   3

[dahtah]

I've never used the magick package, but from what you're saying a "magick-image" object can actually be a collection of images of different sizes? Wouldn't it make more sense then to convert magick objects to image lists? Your point about the alpha channel is well-taken. How about returning it as an attribute?

"as.cimg.magick-image" <- function(im)
{
    out <- map_il(seq_len(length(im)), ~ cvt.frame(im[[.]])) %>% imappend("z")
    alpha <- imsub(out,cc==4)
    out <- imsub(out,cc<=3)
    attr(out,"alpha") <- alpha
    out
}

[jwijffels]

This looks fine to me! Or wait. I wasn't expecting this to happen if we change as.double to as.integer

cvt.frame <- function(f){
  f <- as.double(f)
  d <- dim(f)
  dim(f) <- c(d[1:2],1,4)
  cimg(f) %>% imrotate(90) %>% mirror("x")
}
bigdata <- image_read('https://jeroenooms.github.io/images/bigdata.jpg')
as.cimg(bigdata) %>% range
[1] 0 1

cvt.frame <- function(f){
  f <- as.integer(f)
  d <- dim(f)
  dim(f) <- c(d[1:2],1,4)
  cimg(f) %>% imrotate(90) %>% mirror("x")
}
bigdata <- image_read('https://jeroenooms.github.io/images/bigdata.jpg')
as.cimg(bigdata) %>% range
[1]   0 255

[dahtah]

The difference between as.integer and as.double is magick's doing, it basically assumes different scalings for double and integer. imager uses doubles by default (with 0-1 scaling), so I'd stick with as.double.

[jwijffels]

Sorry about the late reply (was busy creating a course on computer vision). Yes, this is magick doing this. Requested this here: https://github.com/ropensci/magick/issues/47 and it is clear from the code at https://github.com/ropensci/magick/blob/master/R/base.R. But I wonder if this is correct. values of RGB values of 255 never get 1.

[dahtah]

The current master branch holds two functions called magick2cimg and magick2imlist, implementing the conversions. They're not called "as.cimg.magick-image" and "as.imlist.magick-image" because CRAN throws a spurious error (there's a bug in CRAN check).

[jwijffels]

This is great! And it is on CRAN already. Thank you. This simplified some code in some image recognition packages which I recently created (https://github.com/bnosac/image)

[jwijffels]

I don't know what changed in imager/magick but this no longer works:

> img <- image_read(system.file(package = "imager", "extdata", "Leonardo_Birds.jpg"))
> magick2cimg(img)
Error in dim(f) <- c(d[1:2], 1, 4) : 
  dims [product 818960] do not match the length of object [614220]

[dahtah]

Thanks for reporting this. My guess is that something's changed in magick rather than imager. Will investigate.

[dahtah]

Should be fixed in the current master branch

[jwijffels]

many thanks!