Make null device configurable instead of using pdf()?

[clauswilke]

The function gridGraphics::grid.echo.function opens a pdf device: https://github.com/pmur002/gridgraphics/blob/af5078f9e97ac30981f5a198330b4fa750c9bb30/R/graphics.R#L88 This prevents gridGraphics from working in scenarios that pdf() can't handle, specifically non-standard fonts in OS X or Windows. I've come across the same problem in my package cowplot, e.g. this issue. The only solution that I've found to work reliably is one where users can choose which null device to use, because any given choice will break in some scenario.

Here is an example of how the current gridGraphics implementation breaks on my system:

# make function that creates plot with non-standard font
pfun <- function() {
  par(family = "Comic Sans MS")
  plot(1:10, 1:10)
}

pfun() # works, e.g. in R Studio on OS X

gridGraphics::grid.echo(pfun) # doesn't work
## There were 25 warnings (use warnings() to see them)

The warnings show that the problem is the internal use of the pdf device, which looks for PostScript fonts:

> warnings()
Warning messages:
1: In axis(side = side, at = at, labels = labels, ...) :
  font family 'Comic Sans MS' not found in PostScript font database
2: In axis(side = side, at = at, labels = labels, ...) :
  font family 'Comic Sans MS' not found in PostScript font database
3: In axis(side = side, at = at, labels = labels, ...) :
  font family 'Comic Sans MS' not found in PostScript font database
...

I want to emphasize that recordPlot itself has no problems with this font, and neither has grid. For example, the following works just fine:

pfun() # call the plot function interactively

p <- recordPlot() # record previous plot

# now make ggplot2 plot using this font
library(ggplot2)
ggplot(iris, aes(Sepal.Length, Sepal.Width, color=Species)) + 
  geom_point() + theme_minimal(base_family = "Comic Sans MS")

p # print previously recorded plot

However, grid.grabExpr() also opens a pdf device, so that would have to be modified as well for a complete workflow that captures base graphics and turns them into grobs.

> grid.grabExpr
function (expr, warn = 2, wrap = FALSE, ...) 
{
    pdf(file = NULL)
    on.exit(dev.off())
    eval(expr)
    grabDL(warn, wrap, ...)
}

[clauswilke]

Actually, the init() function has the same problem: https://github.com/pmur002/gridgraphics/blob/640e551e0feadb9234150b1c7e41094d63d4a6e9/R/core.R#L16

[pmur002]

Thanks very much for the report and analysis of the problem. IIRC the main reason for the use of pdf(NULL) was to be able to draw off-screen (but not have to touch the file system either). At first glance, adding a 'device' argument to a couple of functions might be all that's needed, though using a non-pdf(NULL) device will, at least in some cases, result in graphics windows appearing and disappearing. I assume that's a price you would be willing to pay ?

[clauswilke]

Yes, I've resigned myself to being willing to pay this price, though I think it's best to use devices that write files rather than open windows. This is an issue I've been struggling with for over a year now, and I've come to the conclusion that there is no graphics device in R that can reliably draw off screen at all times. pdf(NULL) has problems with fonts. Cairo(type = "raster") works but may not be available, and it has font bugs in OS X. png() works great in OS X but creates annoying files.

I think pdf(NULL) is a good default option, but allowing users who know what they're doing to switch this out for something else can be really helpful. Also, downstream applications could remove the annoying files before the user ever sees them.

[pmur002]

Cool. Then I will explore adding a 'device' option to grid.echo() and see how that goes. I will also call on you for testing if that's ok.

[clauswilke]

Yes, absolutely, happy to review and test things.

[pmur002]

Great. Thanks. I have pushed some changes. Here is a new test to try ...

## An example where font metrics matter ...

test <- function() {
    par(family = "Comic Sans MS")
    plot.new()
    text(.5, .5, "LARGE", cex=10)
    w <- strwidth("LARGE", cex=10)
    h <- strheight("LARGE", cex=10)
    rect(.5 - w/2, .5 - h/2, .5 + w/2, .5 + h/2)
}

## Rectangle should wrap text (pretty) nicely
test()

## Rectangle should be wrong size
gridGraphics::grid.echo(test) 

## Solution is to be able to specify device (which can handle original font)
## (at the price of having a device blip up)

## On-screen device
onscreen <- function(w, h) {
    x11(width=w, height=h)
}
gridGraphics::grid.echo(test, device=onscreen) 

## File system device
filesystem <- function(w, h) {
    png(width=w, height=h, units="in", res=96)
    dev.control("enable")
}
gridGraphics::grid.echo(test, device=filesystem) 

Please let me know if that works for you.

[clauswilke]

Yes, this seems to work. Thanks a lot!

There are some issues with your specific example in interactive use, but I think they are due to how R Studio works, not due to gridGraphics. R Studio seems to provide the wrong device widths and heights. Things work fine interactively from the command prompt, and also everything works great inside R Markdown. Finally, not getting all the warnings is also good.

Is it possible to make the same change to grid::grid.grabExpr(), so I can convert R base graphics into grobs I can work with?

[pmur002]

Excellent. Thanks for confirming. I'll have a look at grid::grid.grabExpr().

[pmur002]

Would you mind providing me with a sample piece of code showing how you would like to use grid::grid.grabExpr(), so that I can make sure I am covering your use case ?

I am adding a 'width' and 'height' argument to grid.grabExpr() as well so that you can control the size of the "off screen" device that you temporarily draw into.

[pmur002]

I have committed my changes to grid.grabExpr() to the development version of R (to meet the impending deadline for R 3.5.0). Are you able to test that ? I would still be interested in an example use if you can supply one.

[clauswilke]

Thanks! I thought I posted an example yesterday, but now I don't see it here in this thread. I wonder what happened. I will re-create one.

What's the deadline for R 3.5.0? I'm sure I can set up a development environment to test, but I don't have it running right now and I may not get around to setting it up for a while.

[clauswilke]

Here is an example. In general, I want to combine base graphics and grid graphics into a single grid-based plot. I'm using grid.arrange() here for the final combination, but in practice it could be any function that works with grobs.

library(ggplot2)
library(grid)
library(gridGraphics)
library(gridExtra)

pfun <- function() {plot(1:10, 1:10)}
p2 <- ggplot(iris, aes(Sepal.Width, Sepal.Length)) + geom_point()

g <- grid.grabExpr(grid.echo(pfun))

grid.arrange(g, p2, nrow = 1)

See also the section on supported plot formats here.

[clauswilke]

I installed R devel, and it looks like things work as expected:

library(grid)
library(gridGraphics)
library(ggplot2)

pfun <- function() {
  par(xpd = NA, # switch off clipping, necessary to always see axis labels
    bg = "transparent", # switch off background to avoid obscuring adjacent plots
    oma = c(2, 2, 0, 0), # move plot to the right and up
    mgp = c(2, 1, 0), # move axis labels closer to axis
    family = "Comic Sans MS" # comic sans font
  ) 
  plot(1:10, 1:10)
}

p2 <- ggplot(iris, aes(Sepal.Width, Sepal.Length)) + 
  geom_point() +
  theme_minimal(base_family = "Comic Sans MS")

filesystem <- function(w, h) {
    png(width=w, height=h, units="in", res=96)
    dev.control("enable")
}

g <- grid.grabExpr(grid.echo(pfun, device = filesystem), device = filesystem)

# arrange with gridExtra::grid.arrange
gridExtra::grid.arrange(g, p2, nrow = 1)

# arrange with cowplot::plot_grid
cowplot::plot_grid(g, p2, labels = "auto", label_fontfamily = "Comic Sans MS")

[clauswilke]

However, things do not work with the plot3D library. Not sure what the problem is. I don't get any warnings or errors, but the fonts revert to default.

library(grid)
library(gridGraphics)
library(plot3D)
pfun2 <- function() {
  par(xpd = NA,
    bg = "transparent",
    mar = c(0, 0, 0, 0),
    mai = c(0, 0.1, 0, 0),
    family = "Comic Sans MS" # comic sans font
  )
  scatter3D(mtcars$disp, mtcars$hp, mtcars$mpg, colvar = mtcars$cyl,
            pch = 19, bty ="b2", theta = 20, phi = 30, colkey = FALSE, 
            xlab = "displacement (cu. in.)", ylab ="power (hp)", zlab = "efficiency (mpg)")
}

# this works
pfun2()

# this does not work
grid.echo(pfun2, device = filesystem)

[pmur002]

I have been doing some testing based on your example (thanks for that!) and it looks to be working pretty well (see the attached script - you will need an even newer r-devel for the recordGrob() solution, or change to grid:::recordGrob()).

ClausWilke.txt

Will look at the 'plot3D' problem. Intriguing ...

[pmur002]

I see the problem; the code for echoing persp() plots does not capture the font family. Will get to work on a fix.

[pmur002]

Fix pushed (that works for me anyway).

[clauswilke]

Yes, fonts in the 3d plot work for me too now.

One other thing that I noticed with the 3d plot: It has a fixed aspect ratio using base graphics but it doesn't after grid.echo():

pfun2()
# output after resizing plot window

grid.echo(pfun2, device = filesystem)
# output after resizing plot window

[clauswilke]

I also looked into the recordGrob() option. (Btw., I updated from svn, but still recordGrob() wasn't exported. Can you check whether you committed that change?)

It works great, until you resize the window to a size where the plot doesn't fit. Then, this error is printed out:

Error in plot.new() : figure margins too large

and subsequently R locks up hard and does not recover. (On my Mac, I get the endless spinning rainbow.) I need to close the terminal in which it is running to terminate the process.

[clauswilke]

One more comments: Option 4, with g4 <- grob(cl="delayed"), also breaks when the window is resized to a size where the plot doesn't fit. However, now instead of a lock-up R actually crashes.

> grid.arrange(g4, p2, nrow = 1)
> Mar 21 23:34:12  R[26471] <Error>: CGContextTranslateCTM: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.
Mar 21 23:34:12  R[26471] <Error>: CGContextScaleCTM: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.
Error in plot.new() : figure margins too large
Mar 21 23:34:12  R[26471] <Error>: CGBitmapContextCreateImage: invalid context 0x0. If you want to see the backtrace, please set CG_CONTEXT_SHOW_BACKTRACE environmental variable.
Trace/BPT trap: 5

[pmur002]

The resize issue is a known limitation (see "Limitations" in https://journal.r-project.org/archive/2015/RJ-2015-012/RJ-2015-012.pdf). Some of the examples I sent in the last script demonstrate some of the workarounds.

The lock up and crash is less desirable. Looks like it might be a Cairo device issue though so I will try to replicate on my set up.

[clauswilke]

FYI, I didn't use Cairo. I used the OS X png quartz device.

[pmur002]

Right. Thanks. My mistake. "CG" stands for (Quartz) Core Graphics, not Cairo Graphics :)

Unfortunately, that's going to make it hard for me to debug.

[pmur002]

I have pushed a couple of changes to try to survive the resize problems better (by not leaving the "off-screen" device open when an error occurs during echoing). Does that improve either of the resize scenarios on your system ?

[clauswilke]

The crash is gone but now both approaches cause R to hang and I need to kill the terminal. I'm at SVN revision r74446 and I'm running the following simplified code:

library(ggplot2)
library(grid)
library(gridGraphics)
library(gridExtra)

pfun <- function() {
    plot(1:10, 1:10)
}

p2 <- ggplot(iris, aes(Sepal.Width, Sepal.Length)) + geom_point()
g3 <- grid:::recordGrob(grid.echo(pfun, newpage=FALSE), list())
grid.arrange(g3, p2, nrow = 1) # R hangs if plot window is made too small

g4 <- grob(cl="delayed")
drawDetails.delayed <- function(x, ...) {
    grid.echo(pfun, newpage=FALSE)
}
grid.arrange(g4, p2, nrow = 1)  # R hangs if plot window is made too small

[pmur002]

Progress! Of a sort. I can't hang R, but I can get readline into an unhappy state, which makes me think this might be an event handling problem (windowing system versus R repl versus anything else that is trying to listen to the user). Very bad.

The best advice for now remains: just don't do that (resize a graphics window that will trigger other graphics devices to be rapidly created and destroyed).

[clauswilke]

Yeah, I was hoping to include this in my library to make it easier to mix base plots with grid. But I think any option that might routinely lock up R is not viable.

I tried to catch the error when it happens, like so:

g3 <- recordGrob(tryCatch(grid.echo(pfun, newpage=FALSE), error = function(e) {}), list())

but then I get a new error:

Error in UseMethod("depth") : 
  no applicable method for 'depth' applied to an object of class "NULL"

So it seems that somehow the graphics device onto which we're echoing disappears or goes into an undefined state, even though the shutdown() function should restore it.

[clauswilke]

Actually, ignore the last comment. I had reverted to an earlier version of gridGraphics for some reason. The tryCatch() construct seems to work! I'll test some more later today.

[clauswilke]

Ok, things seem to work correctly now.

Putting everything together in the way I plan to use it:

library(grid)
library(gridGraphics)
library(plot3D)
library(ggplot2)

# 2d base plot
pfun <- function() {
    par(family = "Arial Narrow")
    plot(1:10, 1:10)
}

# 3d base plot
pfun2 <- function(theta = 20, phi = 30) {
  function() {
    par(xpd = NA,
        bg = "transparent",
        mar = c(0, 0, 0, 0),
        mai = c(0, 0.1, 0, 0),
        family = "Arial Narrow"
    )
    scatter3D(mtcars$disp, mtcars$hp, mtcars$mpg, colvar = mtcars$cyl,
              pch = 19, bty ="b2", theta = theta, phi = phi, colkey = FALSE, 
              xlab = "displacement (cu. in.)", ylab ="power (hp)", zlab = "efficiency (mpg)")
  }
}

# grid-based plot
p <- ggplot(iris, aes(Sepal.Width, Sepal.Length)) + geom_point() +
       theme_minimal(base_family = "Arial Narrow")

# png null device
filesystem <- function(w, h) {
  png(width=w, height=h, units="in", res=96)
  dev.control("enable")
}

# function that reliably captures a base plot and turns it into a grob
plot_to_grob <- function(plotfun) {
  recordGrob(tryCatch(grid.echo(plotfun, newpage=FALSE, device = filesystem),
             error = function(e) {}),
             list(plotfun = plotfun))
}

# combine base plot and grid-based plot
g <- plot_to_grob(pfun)
cowplot::plot_grid(g, p, labels = "auto", label_fontfamily = "Arial Narrow")

Interactive resizing works:

Errors are handled gracefully; the plot simply disappears when the plot area is too small, and it reappears once this is rectified.

Now combining 3d plots into a grid:

g1 <- plot_to_grob(pfun2(30, 20))
g2 <- plot_to_grob(pfun2(-30, 20))
g3 <- plot_to_grob(pfun2(30, 40))
g4 <- plot_to_grob(pfun2(-30, 40))

cowplot::plot_grid(g1, g2, g3, g4, labels = "auto", label_fontfamily = "Arial Narrow")

Again, resizing works (not shown).

[pmur002]

Great! I will try to get the new version of gridGraphics onto CRAN

[clauswilke]

Thanks a lot for your quick responses to all my comments! From my perspective, this issue can be closed.