feature request: geom_image that allows plotting of several images, based on geom_rect?

[japhir]

As discussed on twitter, it would be great if cowplot would allow plotting of several images in one geom_* call, while listening to the xmin, xmax, ymin, ymax paramters that are used in geom_rect.

Something along the lines of:

library(ggplot2)
library(cowplot)
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union
library(tibble)

dat <- tribble(~ left, ~right, ~bot, ~ top, ~url,
               1.5, 3, 0, 5, "https://upload.wikimedia.org/wikipedia/commons/thumb/0/0b/Cat_poster_1.jpg/390px-Cat_poster_1.jpg",
               3, 7, 2, 6, "https://upload.wikimedia.org/wikipedia/commons/thumb/6/66/An_up-close_picture_of_a_curious_male_domestic_shorthair_tabby_cat.jpg/800px-An_up-close_picture_of_a_curious_male_domestic_shorthair_tabby_cat.jpg",
               7, 10, 4, 8, "https://upload.wikimedia.org/wikipedia/commons/thumb/4/4d/Cat_November_2010-1a.jpg/800px-Cat_November_2010-1a.jpg")

dat %>%
  ggplot(aes(xmin = left, xmax = right, ymin = bot, ymax = top)) +
  geom_rect(fill = "gray", colour = "black") +
  # desired way
  ## geom_image(aes(image = img), interpolate = FALSE)
  # hacky way to get output
  cowplot::draw_image(dat$url[[1]],
                      x = dat$left[[1]],
                      y = dat$bot[[1]],
                      width = dat$right[[1]] - dat$left[[1]],
                      height = dat$top[[1]] - dat$bot[[1]], interpolate = FALSE) +
  cowplot::draw_image(dat$url[[2]],
                      x = dat$left[[2]],
                      y = dat$bot[[2]],
                      width = dat$right[[2]] - dat$left[[2]],
                      height = dat$top[[2]] - dat$bot[[2]], interpolate = FALSE) +
  cowplot::draw_image(dat$url[[3]],
                      x = dat$left[[3]],
                      y = dat$bot[[3]],
                      width = dat$right[[3]] - dat$left[[3]],
                      height = dat$top[[3]] - dat$bot[[3]], interpolate = FALSE)

^{Created on 2020-11-07 by the reprex package (v0.3.0)}

[japhir]

I'll go ahead and mention the issue over at ggimage here, so that it can cross-link for further discussion: https://github.com/GuangchuangYu/ggimage/issues/6

[japhir]

In the geom_image call, I think that the images should stretch by default, unless some scale parameter is set manually to prevent this from happening (something like preserve_aspect_ratio?)

[clauswilke]

I disagree on the last point. Images should be shown at their native aspect ratio unless explicitly requested not to. It's extremely rare that you would want to reproduce an image at a modified aspect ratio. That's why draw_image() behaves the way it does. I think your example shows that it's the correct behavior.

[japhir]

I guess that makes sense for most use-cases.

Would implementing a stretching parameter be difficult? For my use-case we have very thin core photographs, and usually we have to plot many on top of each other at specific "depth" values. This means that you'd see only a few pixels if you put them at the correct aspect ratio, so usually they are stretched horizontally.

[japhir]

I updated the reprex to double-check that images at the original aspect ratio are currently constrained to the bounding box by whichever dimension is smaller. Seems to work as intended!

[clauswilke]

I just realized that what you want should be easily possible with the ggtextures package: https://github.com/clauswilke/ggtextures

[japhir]

Hmm those seem to be targetting how to tile images, does that cost any overhead when you set nrow/ncol to 1? I think I managed to get it to work as I desire:

library(ggplot2)
library(tibble)
library(ggtextures)

data <- tibble(
  xmin = c(1, 2.5), ymin = c(1, 1), xmax = c(2, 4), ymax = c(4, 3),
  fill = c("a", "b"),
  image = list(
    "https://jeroen.github.io/images/Rlogo.png",
    magick::image_read_svg("https://jeroen.github.io/images/tiger.svg")
  )
)

ggplot(data, aes(xmin = xmin, xmax = xmax,
                 ymin = ymin, ymax = ymax,
                 fill = fill, image = image)) +
  geom_textured_rect(nrow = 1,
                     ncol = 1,
                     img_width = unit(1, "null"),
                     img_height = unit(1, "null"),
                     position = "identity")

^{Created on 2020-11-08 by the reprex package (v0.3.0)}

[japhir]

Oh and I didn't see the interpolate option. Is that on the roadmap for the dev package?

[clauswilke]

Your reprex is exactly how I would use ggtextures, and there is no meaningful overhead from setting ncol and nrow to 1. In fact, it's likely less overhead than any possible cowplot solution would have.

Not sure what you mean by interpolate option. If you set both img_width and img_height to unit(1, "null") the images are stretched to exactly fill out the rectangles provided. Isn't that what you wanted?

[japhir]

Great! Then it looks like I understood it! :)

I do want the images to stretch as they do, but since the images are quite big I don't want imagemagick to do any interpolation when plotting the points. Cowplot's draw_image has this paramter:

interpolate: A logical value indicating whether to linearly interpolate
          the image (the alternative is to use nearest-neighbour
          interpolation, which gives a more blocky result).

and I didn't see this in the ggtextures package.

Thanks for getting back to me so quickly and helping me find a solution, by the way :+1:

[clauswilke]

I see. Please open an issue for the interpolate option in the ggtextures repo. It would be pretty straightforward to add this, it's just an optional argument to rasterGrob that would have to be collected and carried through.

[japhir]

Great, I guess we can close this issue as well then!

Here's what I was able to do with it: https://github.com/japhir/corepics