Project: Adding Alpha Masks to the Quartz Graphics Device

[hturner]

Discussed in https://github.com/r-devel/r-project-sprint-2023/discussions/2

^{Originally posted by **giscus[bot]** June 1, 2023} # R Project Sprint 2023 - Adding Alpha Masks to the Quartz Graphics Device https://contributor.r-project.org/r-project-sprint-2023/projects/quartz-alpha-mask/

[georgestagg]

Amusingly the simple diff below seems to be enough to get it to work, at least on my version of macOS.

I checked the docs for CGContextClipToMask, and the documentation does clearly state that the mask image should be in grayscale without an alpha channel, so even if this works it is not "correct". I guess it is simply luck and/or undocumented that clipping to an RGBA mask image also just works.

Tomorrow I will take a stab at converting the resulting RGBA image to a grayscale context/image and passing that to CGContextClipToMask() instead.

Index: src/library/grDevices/src/devQuartz.c
===================================================================
--- src/library/grDevices/src/devQuartz.c   (revision 84998)
+++ src/library/grDevices/src/devQuartz.c   (working copy)
@@ -1030,7 +1030,7 @@
         QMaskRef quartz_mask = malloc(sizeof(QMaskRef));
         if (!quartz_mask) error(_("Failed to create Quartz mask"));

-        cs = CGColorSpaceCreateDeviceGray();
+        cs = CGColorSpaceCreateDeviceRGB();

         /* Create bitmap grahics context 
          * drawing is redirected to this context */
@@ -1040,7 +1040,7 @@
                                               8,
                                               0,
                                               cs,
-                                              kCGImageAlphaNone);
+                                              kCGImageAlphaPremultipliedLast);

         quartz_mask->context = quartz_bitmap;
         xd->masks[index] = quartz_mask;
@@ -2715,10 +2715,6 @@
     if (isNull(mask)) {
         /* Set NO mask */
         index = -1;
-    } else if (R_GE_maskType(mask) == R_GE_alphaMask) {
-        warning(_("Ignored alpha mask (not supported on this device)"));
-        /* Set NO mask */
-        index = -1;        
     } else {
         if (isNull(ref)) {
             /* Create a new mask */

[pmur002]

Nice discovery!

Presumably that simple patch would break luminance masks though?

And on that note, if you can get something going that agrees with the documentation, it will be important to check that your patch does not break existing behaviour. See #74 for a discussion of some of the checks that would be useful, in particular the 'gdiff' testing. There are some tests for masks in grid/tests/masks.R.

Just looking at those mask tests prompts a couple more thoughts:

• the tests actually choose what type of mask to use based on the capabilities of the device ; it would be nicer if the tests checked both types of mask if they are supported.

• the RQuartz_capabilities() would need updating.

Thanks for taking a look at this!

[georgestagg]

For Paul (and anyone else following along with the sprint):

I ended up mostly looking at other things today, but did make minor progress. The latest version of the patch is below, and sample output also attached (for both mask types).

I still don't think this is quite following the letter of the CGContextClipToMask() docs, but hopefully I'll get a chance to continue to hack at it some more tomorrow. Will also attempt to take a look at tests and gdiff tomorrow too if I make further progress.

Index: src/library/grDevices/src/devQuartz.c
===================================================================
--- src/library/grDevices/src/devQuartz.c   (revision 85035)
+++ src/library/grDevices/src/devQuartz.c   (working copy)
@@ -1031,7 +1031,13 @@
         if (!quartz_mask) error(_("Failed to create Quartz mask"));

         cs = CGColorSpaceCreateDeviceGray();
-        
+
+        /* Setup bitmap info for the type of masking */
+        uint32_t bitmapInfo = kCGImageAlphaOnly;
+        if (R_GE_maskType(mask) == R_GE_luminanceMask) {
+            bitmapInfo = kCGImageAlphaNone;
+        }
+
         /* Create bitmap grahics context 
          * drawing is redirected to this context */
         quartz_bitmap = CGBitmapContextCreate(NULL,
@@ -1040,7 +1046,7 @@
                                               8,
                                               0,
                                               cs,
-                                              kCGImageAlphaNone);
+                                              bitmapInfo);

         quartz_mask->context = quartz_bitmap;
         xd->masks[index] = quartz_mask;
@@ -2715,10 +2721,6 @@
     if (isNull(mask)) {
         /* Set NO mask */
         index = -1;
-    } else if (R_GE_maskType(mask) == R_GE_alphaMask) {
-        warning(_("Ignored alpha mask (not supported on this device)"));
-        /* Set NO mask */
-        index = -1;        
     } else {
         if (isNull(ref)) {
             /* Create a new mask */
@@ -2938,8 +2940,9 @@
     SET_VECTOR_ELT(capabilities, R_GE_capability_clippingPaths, clippingPaths);
     UNPROTECT(1);

-    PROTECT(masks = allocVector(INTSXP, 1));
+    PROTECT(masks = allocVector(INTSXP, 2));
     INTEGER(masks)[0] = R_GE_luminanceMask;
+    INTEGER(masks)[1] = R_GE_alphaMask;
     SET_VECTOR_ELT(capabilities, R_GE_capability_masks, masks);
     UNPROTECT(1);

[pmur002]

Cool. Nice output!

So this is still taking advantage of the undocumented behaviour, right? Yes, it would be ideal to avoid that, possibly by drawing the mask onto an RGBA image and then constructing a greyscale image from the alpha channel of the RGBA image.

I think effort in that direction would be more useful than 'gdiff'ing the current "naughty" solution. I would rather have an untested legal solution than a tested illegal one, if that makes sense.

Also, if you get time, please check that the output from dev.capabilities() (on a quartz() device) looks right.

Thanks!

[georgestagg]

The latest version of the patch is below.

In this version, when an alpha mask is used a bitmap with an alpha-only channel is created. When a luminance mask is used instead, the bitmap is a grayscale bitmap with no alpha channel (as before).

When luminance masking, things proceed as in the current R-devel.

When alpha masking, a second bitmap is created that's grayscale with no alpha channel, as required for CGContextClipToMask() when following the letter of the docs. The data in the alpha channel of the first bitmap is copied to the grayscale channel of the second channel, then the first (alpha-only) bitmap is released - we no longer need it. The new grayscale bitmap is then used as the mask image.

---

please check that the output from dev.capabilities() (on a quartz() device) looks right.

$masks now includes "alpha", as expected.

> dev.capabilities()$masks
[1] "luminance" "alpha"    

---

We got caught up in building R on some other machines today, so we never got around to testing things for this patch thoroughly. Perhaps if one of us has time in the coming weeks they could run through the output of src/library/grid/tests/masks.R and check the new quartz() against pdf() to confirm things are OK. This should definitely be done before considering merging the patch.

---

Index: src/library/grDevices/src/devQuartz.c
===================================================================
--- src/library/grDevices/src/devQuartz.c   (revision 85035)
+++ src/library/grDevices/src/devQuartz.c   (working copy)
@@ -1032,6 +1032,12 @@

         cs = CGColorSpaceCreateDeviceGray();

+        /* For alpha masks, create a bitmap with only an alpha channel */
+        uint32_t bitmapInfo = kCGImageAlphaNone;
+        if (R_GE_maskType(mask) == R_GE_alphaMask) {
+            bitmapInfo = kCGImageAlphaOnly;
+        }
+
         /* Create bitmap grahics context 
          * drawing is redirected to this context */
         quartz_bitmap = CGBitmapContextCreate(NULL,
@@ -1040,7 +1046,7 @@
                                               8,
                                               0,
                                               cs,
-                                              kCGImageAlphaNone);
+                                              bitmapInfo);

         quartz_mask->context = quartz_bitmap;
         xd->masks[index] = quartz_mask;
@@ -1055,6 +1061,31 @@
         eval(R_fcall, R_GlobalEnv);
         UNPROTECT(1);

+        /* When working with an alpha mask, convert into a grayscale bitmap */
+        if (R_GE_maskType(mask) == R_GE_alphaMask) {
+            CGContextRef alpha_bitmap = quartz_bitmap;
+
+            /* Create a new grayscale bitmap with no alpha channel */
+            int stride = CGBitmapContextGetBytesPerRow(alpha_bitmap);
+            quartz_bitmap = CGBitmapContextCreate(NULL,
+                                                  (size_t) devWidth,
+                                                  (size_t) devHeight,
+                                                  8,
+                                                  stride,
+                                                  cs,
+                                                  kCGImageAlphaNone);
+            quartz_mask->context = quartz_bitmap;
+            
+            void *alpha_data = CGBitmapContextGetData(alpha_bitmap);
+            void *gray_data = CGBitmapContextGetData(quartz_bitmap);
+
+            /* Copy the alpha channel data into the grayscale bitmap */
+            memcpy(gray_data, alpha_data, stride * devHeight);
+
+            /* We're finished with the alpha channel bitmap now */
+            CGContextRelease(alpha_bitmap);
+        }
+
         /* Create image from bitmap context */
         CGImageRef maskImage;
         maskImage = CGBitmapContextCreateImage(quartz_bitmap);
@@ -2715,10 +2746,6 @@
     if (isNull(mask)) {
         /* Set NO mask */
         index = -1;
-    } else if (R_GE_maskType(mask) == R_GE_alphaMask) {
-        warning(_("Ignored alpha mask (not supported on this device)"));
-        /* Set NO mask */
-        index = -1;        
     } else {
         if (isNull(ref)) {
             /* Create a new mask */
@@ -2938,8 +2965,9 @@
     SET_VECTOR_ELT(capabilities, R_GE_capability_clippingPaths, clippingPaths);
     UNPROTECT(1);

-    PROTECT(masks = allocVector(INTSXP, 1));
+    PROTECT(masks = allocVector(INTSXP, 2));
     INTEGER(masks)[0] = R_GE_luminanceMask;
+    INTEGER(masks)[1] = R_GE_alphaMask;
     SET_VECTOR_ELT(capabilities, R_GE_capability_masks, masks);
     UNPROTECT(1);

[nzgwynn]

I worked on some tests for this that may be helpful for some:

library(grid)

HersheyLabel <- function(x, y=unit(.5, "npc")) {
    lines <- strsplit(x, "\n")[[1]]
    if (!is.unit(y))
        y <- unit(y, "npc")–
    n <- length(lines)
    if (n > 1) {
        y <- y + unit(rev(seq(n)) - mean(seq(n)), "lines")
    }
    grid.text(lines, y=y, gp=gpar(fontfamily="HersheySans"))
}

devMask <- function(aMask, lMask) {
    support <- dev.capabilities()$masks
    if (is.character(support)) {
        if ("alpha" %in% support) {
            aMask
        } else {
            if ("luminance" %in% support) {
                as.mask(lMask, type="luminance")
            } else {
                FALSE
            }
        }
    } else {
        FALSE
    }
}

################################################################################
## works
mask_works <- devMask(circleGrob(r=.3, gp=gpar(fill="black")),
                circleGrob(r=.3, gp=gpar(col="white", fill="white")))

pdf("mask_works.pdf")
grid.newpage()
pushViewport(viewport(mask=mask_works))
grid.rect(width=.5, gp=gpar(fill="black"))
popViewport()
HersheyLabel("solid black rectangle with circle mask", y=.1) 
dev.off()               

## Simple mask
alpha_mask <- circleGrob(r=.3, gp=gpar(fill="black"))

pdf("alpha_mask.pdf")
grid.newpage()
pushViewport(viewport(mask=alpha_mask))
grid.rect(width=.5, gp=gpar(fill="black"))
popViewport()
HersheyLabel("solid black rectangle with circle mask", y=.1)
dev.off()

## VERY thin mask
pdf("mask.pdf")
mask <- devMask(circleGrob(r=.3, gp=gpar(fill=NA))
grid.newpage()
pushViewport(viewport(mask=mask))
grid.rect(width=.5, gp=gpar(fill="black"))
popViewport()
HersheyLabel("solid black rectangle with circle BORDER mask", y=.1)
dev.off()

## Multiple grobs mask
pdf("mask.pdf")
mask <- circleGrob(x=1:3/4, y=1:3/4, r=.1, gp=gpar(fill="black"))
grid.newpage()
pushViewport(viewport(mask=mask))
grid.rect(width=.5, gp=gpar(fill="black"))
popViewport()
HersheyLabel("solid black rectangle with three-circle mask", y=.1)
dev.off()

## Mask with gradient on single grob
pdf("mask.pdf")
mask <- circleGrob(gp=gpar(col=NA,fill=radialGradient(c("black",
                                                         "transparent"))))
grid.newpage()
pushViewport(viewport(mask=mask))
grid.rect(width=.5, gp=gpar(fill="black"))
popViewport()
HersheyLabel("solid black rectangle with radial gradient mask", y=.1)
dev.off()

## Mask with gradient on multiple grobs
pdf("mask.pdf")
grid.newpage()
pushViewport(viewport(mask=mask))
grid.rect(x=1:2/3, width=.2, gp=gpar(fill="black"))
popViewport()
HersheyLabel("two solid black rectangles with radial gradient mask", y=.1)
dev.off()

## Mask with clipping path
pdf("mask.pdf")
mask <- gTree(children=gList(rectGrob(gp=gpar(fill="black"))),
                      vp=viewport(clip=circleGrob(r=.4)))
grid.newpage()
pushViewport(viewport(mask=mask))
grid.rect(width=.5, gp=gpar(fill="grey"))
popViewport()
HersheyLabel("rect is half width and filled grey
mask is full rect with circle clipping path
result is half width rect with rounded top and bottom", y=.1)
dev.off()

## Mask with a mask
mask <- gTree(children=gList(rectGrob(gp=gpar(fill="black"))),
                      vp=viewport(mask=circleGrob(r=.4,
                                                  gp=gpar(fill="black"))))
grid.newpage()
pushViewport(viewport(mask=mask))
grid.rect(width=.5, gp=gpar(fill="grey"))
popViewport()
HersheyLabel("rect is half width and filled grey
mask is full rect with circle mask
result is half width rect with rounded top and bottom", y=.1)
dev.off()

## A mask from two grobs, with ONE grob making use of a clipping path
pdf("mask.pdf") 
grid.newpage()
mask <- gTree(children=gList(rectGrob(x=.25, width=.3, height=.8,
                                              gp=gpar(fill="black"),
                                              vp=viewport(clip=circleGrob(r=.4))),
                                     rectGrob(x=.75, width=.3, height=.8,
                                              gp=gpar(fill="black")))))
pushViewport(viewport(mask=mask))
grid.rect(gp=gpar(fill="grey"))
popViewport()
HersheyLabel("mask is two grobs, ONE with its own (circle) clip path
push mask
rect
result is one slice of circle and one rectangle")
dev.off()

## A mask that is equivalent to ...
## A clipping path that itself makes use of a clipping path !?
pdf("mask.pdf")
grid.newpage()
mask <- devMask(rectGrob(gp=gpar(fill="black"),
                         vp=viewport(width=.5, height=.5, clip=circleGrob())))
pushViewport(viewport(mask=mask))
grid.rect(gp=gpar(fill="grey"))
HersheyLabel("mask includes clip path
(clip path is circle)
push mask
rect
small grey circle")
dev.off()

[pmur002]

Thanks very much! This looks like it makes sense. Will try to do some more testing to confirm.