embed_fonts fails

[kohske]

Hi Winston,

embed_fonts fails In my environment.

> pdf("font_plot.pdf", family="Impact", width=4, height=4)
> plot(mtcars$mpg, mtcars$wt, 
+      main = "Fuel Efficiency of 32 Cars",
+      xlab = "Weight (x1000 lb)",
+      ylab = "Miles per Gallon")
> dev.off()
null device 
          1 
> embed_fonts("font_plot.pdf", outfile="font_plot_embed.pdf")

*** Warning: GenericResourceDir doesn't point to a valid resource directory.
               the -sGenericResourceDir=... option can be used to set this.

   **** Error reading a content stream. The page may be incomplete.
   **** File did not complete the page properly and may be damaged.
   **** Warning: File has unbalanced q/Q operators (too many q's)

   **** This file had errors that were repaired or ignored.
   **** The file was produced by: 
   **** >>>> R 2.15.2 <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.

> 

but this modification removes the error and generate an font-embedded pdf successfully.

> embed_fonts2 <- function (file, format, outfile = file, options = "", fontpaths = character(0)) 
+ {
+   if (missing(format)) {
+     suffix <- gsub(".+[.]", "", file)
+     format <- switch(suffix, ps = , eps = "pswrite", pdf = "pdfwrite")
+   }
+   force(outfile)
+   file <- paste("'", file, "'", sep = "")
+   embedFonts(file = file, format = format, outfile = outfile, 
+              fontpaths = fontpaths)
+ }
> embed_fonts2("font_plot.pdf", outfile="font_plot_embed.pdf", fontpaths="/Library/Fonts")

Here is my sessionInfo.

> sessionInfo()
R version 2.15.2 (2012-10-26)
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)

locale:
[1] ja_JP.UTF-8/ja_JP.UTF-8/ja_JP.UTF-8/C/ja_JP.UTF-8/ja_JP.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] extrafontdb_1.0 ggplot2_0.9.3   extrafont_0.13  devtools_0.8   

loaded via a namespace (and not attached):
 [1] colorspace_1.2-0   dichromat_1.2-4    digest_0.6.0       evaluate_0.4.3     grid_2.15.2        gtable_0.1.2       httr_0.2           labeling_0.1      
 [9] MASS_7.3-22        memoise_0.1        munsell_0.4        parallel_2.15.2    plyr_1.8           proto_0.3-10       RColorBrewer_1.0-5 RCurl_1.95-3      
[17] reshape2_1.2.2     Rttf2pt1_1.1       scales_0.2.3       stringr_0.6.2      tools_2.15.2       whisker_0.1       

do you have any idea?

[wch]

Hm, that is strange. The first thing to try is to delete your existing font database by reinstalling the extrafontdb package:

install.packages('extrafontdb')

# Restart R, then re-import fonts
library(extrafont)
font_import()
loadfonts()

If you still have a problem, then my next theory is that it has something to do with Asian font paths.... I wonder if there's something odd in your Fontmap file. Please run this to get the path to that file:

file.path(extrafont:::fontmap_path(), "Fontmap")
# "/Users/winston/R/extrafontdb/fontmap/Fontmap"

If you look inside that file, do you see any strange paths or filenames? For example, here are some lines from my version of that file:

/CMRoman-Bold (/Users/winston/R/fontcm/fonts/outlines/fcmb8a.pfb) ;
/CMRoman-BoldItalic (/Users/winston/R/fontcm/fonts/outlines/fcmbi8a.pfb) ;
/CMRoman-Italic (/Users/winston/R/fontcm/fonts/outlines/fcmri8a.pfb) ;
/CMRoman-Regular (/Users/winston/R/fontcm/fonts/outlines/fcmr8a.pfb) ;
...
/-Keyboard (/System/Library/Fonts/Keyboard.ttf) ;
...
/Georgia-Bold (/Library/Fonts/Georgia Bold.ttf) ;
/Georgia-BoldItalic (/Library/Fonts/Georgia Bold Italic.ttf) ;
/Georgia-Italic (/Library/Fonts/Georgia Italic.ttf) ;
/Impact (/Library/Fonts/Impact.ttf) ;

Also, you may want to check the output of fonttable() to see if anything looks weird there.

[kohske]

Thanks.

The Fontmap file looks fine, and here is the output of fonttable()

> subset(fonttable(), FontName == "Impact")
   package       afmfile                  fontfile FullName FamilyName FontName  Bold Italic Symbol afmsymfile
72      NA Impact.afm.gz /Library/Fonts/Impact.ttf   Impact     Impact   Impact FALSE  FALSE  FALSE         NA

Here is the trace of the cmd in embedFonts:

> trace(embedFonts, quote(print(cmd)), at = 12)
Tracing function "embedFonts" in package "grDevices"
[1] "embedFonts"
> unload(inst("extrafont"))
> library(extrafont)
> embed_fonts("font_plot.pdf", outfile="font_plot_embed.pdf")
Tracing embedFonts(file = file, format = format, outfile = outfile, options = paste(paste("-I",  .... step 12 
[1] "gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=/var/folders/na/na81MsJUHRmVyCHG4xD0X++++TI/-Tmp-//RtmpByAuFz/Rembed53e685a1258 -sFONTPATH= -I'/Library/Frameworks/R.framework/Versions/2.15/Resources/library/extrafontdb/fontmap'  'font_plot.pdf'"

The cmd looks wired.

[1] "gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=/var/folders/na/na81MsJUHRmVyCHG4xD0X++++TI/-Tmp-//RtmpByAuFz/Rembed53e685a1258 -sFONTPATH= -I'/Library/Frameworks/R.framework/Versions/2.15/Resources/library/extrafontdb/fontmap'  'font_plot.pdf'"

especially for -sFONTPATH=

maybe this is the problem by using different version of gs?

> system("gs --version")
9.05
> system("gs --help")
GPL Ghostscript 9.05 (2012-02-08)
Copyright (C) 2010 Artifex Software, Inc.  All rights reserved.

[kohske]

And this command (from shell) can generate the embedded pdf.

gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=/var/folders/na/na81MsJUHRmVyCHG4xD0X++++TI/-Tmp-//RtmpukJoUm/Rembed64d5d8cefdf -sFONTPATH=/Library/Fonts  'font_plot.pdf'

then, this induces the same error:

gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=/var/folders/na/na81MsJUHRmVyCHG4xD0X++++TI/-Tmp-//RtmpukJoUm/Rembed64df005022 -sFONTPATH=/Library/Fonts -I'/Library/Frameworks/R.framework/Versions/2.15/Resources/library/extrafontdb/fontmap'  'font_plot.pdf'

So, -I'/Library/Frameworks/R.framework/Versions/2.15/Resources/library/extrafontdb/fontmap' is doing something wired.

[wch]

My cmd looks the same:

> embed_fonts("font_plot.pdf", outfile="font_plot_embed.pdf")
Tracing embedFonts(file = file, format = format, outfile = outfile, options = paste(paste("-I",  .... step 12 
[1] "gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=/var/folders/26/df_1sr1d2w1_x179whkc7xxw0000gn/T//Rtmp4E8FlX/Rembedc0f0441dd951 -sFONTPATH= -I'/Users/winston/R/extrafontdb/fontmap'  'font_plot.pdf'"

The -I option points it to the Fontmap file, and that file contains the font path for each font. I still wonder if your Fontmap file has something strange in it. Here is another experiment to try (two options):

• Delete every line in Fontmap except the line for Impact.
• Re-install extrafontdb, and then run font_import(pattern="Impact").

[kohske]

Still not fixed. Could you please show the output of gs -h?

[kohske]

Just in case this is the content of Fontmap:

/Impact (/Library/Fonts/Impact.ttf) ;

now there is only 1 line.

[kohske]

Finally I may figure out the reason. It may be the format of path.

[takahashi@haruna extrafont]$ gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -I/Users/takahashi/tmp/fontmap -sDEVICE=pdfwrite -sOutputFile=f2.pdf 'font_plot.pdf'
[takahashi@haruna extrafont]$ gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -I/Library/Frameworks/R.framework/Versions/2.15/Resources/library/extrafontdb/fontmap -sDEVICE=pdfwrite -sOutputFile=f2.pdf 'font_plot.pdf'

*** Warning: GenericResourceDir doesn't point to a valid resource directory.
               the -sGenericResourceDir=... option can be used to set this.

The contents of the fontmap folder is same. I couldn't find any reference of the format of the path for gs resource dir, but only difference is the name of path...

Probably this is less related to the extrafont itself.

[kohske]

Finally I figured out. The -I options cannot correctly handle the path including the string "Resources"

[takahashi@haruna extrafont]$ gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -I/Users/takahashi/tmp/Resources/x -sDEVICE=pdfwrite -sOutputFile=f2.pdf 'font_plot.pdf'

*** Warning: GenericResourceDir doesn't point to a valid resource directory.
               the -sGenericResourceDir=... option can be used to set this.

gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -I/Users/takahashi/tmp/zesources/x -sDEVICE=pdfwrite -sOutputFile=f2.pdf 'font_plot.pdf'
[takahashi@haruna extrafont]$ gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -I/Users/takahashi/tmp/zesources/x -sDEVICE=pdfwrite -sOutputFile=f2.pdf 'font_plot.pdf'
[takahashi@haruna extrafont]$ 

Could you please test this? This is definitely the bug of gs..., but the R library path in OSX includes "Resources", some dirty hack may be necessary.

[wch]

Wow, I think you're right:

$ gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=out.pdf -sFONTPATH= -I'/Users/winston/temp/Resources/fontmap'  'font_plot.pdf'

*** Warning: GenericResourceDir doesn't point to a valid resource directory.
               the -sGenericResourceDir=... option can be used to set this.

   **** Error reading a content stream. The page may be incomplete.
   **** File did not complete the page properly and may be damaged.
   **** Warning: File has unbalanced q/Q operators (too many q's)

   **** This file had errors that were repaired or ignored.
   **** The file was produced by: 
   **** >>>> R 2.15.2 <<<<
   **** Please notify the author of the software that produced this
   **** file that it does not conform to Adobe's published PDF
   **** specification.

$ gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=out.pdf -sFONTPATH= -I'/Users/winston/temp/esources/fontmap'  'font_plot.pdf'

Here's a hack that seems to work for me: use resources instead of Resources:

gs -dNOPAUSE -dBATCH -q -dAutoRotatePages=/None -sDEVICE=pdfwrite -sOutputFile=out.pdf -sFONTPATH= -I'/Users/winston/temp/resources/fontmap'  'font_plot.pdf'

Yuck. But it works. (Assuming a case-insensitive HFS filesystem)

[kohske]

Yes, resource works also for me. Thanks.

Then, should the fixpath_os like this?

function (path) 
{
    if (grepl("^mingw", sessionInfo()$R.version$os)) {
        gsub("/", "\\\\", path)
    } elseif (grepl("darwin", R.version$os) {
        gsub("Resources", "resources")
    }
    else {
        path
    }
}

[wch]

I think something like that should work, but it should be done outside of that function, and inside embed_fonts instead. This is because fixpath_os should be usable in other places where the Resources hack is unnecessary.

[wch]

Hm, now that I think about it a bit more, it might be better to create a symlink to the fontmap directory in the temp dir. This is because the resources hack won't work for computers with case-sensitive HFS+ filesystems.

[kohske]

Hm, now that I think about it a bit more, it might be better to create a symlink to the fontmap directory in the temp dir.

agreed. Then, in embed_fonts,

1. copy the Fontmap into a temp dir.
2. run embedding with fontmap of the temp dir
3. remove the Fontmap in the temp dir.

make sense.

[wch]

OK, I pushed a fix in the fix-fontmap branch, but I haven't tested it. Can you try it and tell me if it works?

[kohske]

Excellent, works perfectly!!

Thanks.

[wch]

Thanks for your help figuring out the problem and a fix!

I made a small change and merged it into master. Can you test the master branch?

[kohske]

The test on the master branch is ok!! Thanks for your excellent work!

[wch]

Great, thank you!