color issues: RGB -> CMYK

[friendly]

In the instructions with the copy edit, we are asked to make sure that all color in the book uses CMYK rather than RGB colors.

Back in April, Marcus did a test using Adobe, and reported: ''That looks good for the most part. However there are still some issues with RGB and the font embedding. Attached is a “preflight” report I created using Adobe Acrobat Pro for those problems. (If you have Acrobat Pro I can explain how to generate this yourself if you’d like.) Each instance is highlighted with a pdf comment; there are 5 pages which have RGB.'' (See the file ChapmanHall/chapter05_report.pdf)

@MFont : Could you do another test on the recently compiled versions of chapter01-06.pdf?

I think, but am not sure that (a) all LaTeX code (e.g., tikz) now uses CMYK, via \usepackage[cmyk]{xcolor} I think this also applies to the lines in inputs/knitr-theme-mods.tex which set the highlighting for chunks, via lines like

\definecolor{fgcolor}{rgb}{0.345, 0.345, 0.345}
\renewcommand{\hlnum}[1]{\textcolor[rgb]{0.686,0.059,0.569}{#1}}%                numbers
 ...

(b) all R figures produced inline by knitr now use CMYK via grDevices::pdf.options(colormodel = "cmyk") in Rprofile.R

This leaves the figures that are now included directly by \includegraphics[]{}: (c) Figures and diagrams produced largely by @mattsigal using Adobe Illustrator or other software (d) Figures that were produced originally in R, but "frozen" in the book for various reasons (e.g., manual tweaks, clipping to avoid large margins, etc.)

How to handle these? One possibility that occurs to me is to use ImageMagik convert, along the lines

convert fig-rgb.pdf -colorspace CMYK fig-cmyk.pdf

but I don't know how successful this is likely to be. There is some discussion at http://www.imagemagick.org/discourse-server/viewtopic.php?t=16572

[friendly]

A quick check, via tcgrep -E Rnw '\\includegraphics' shows that there are 38 such figs. There is a list of them in the file includegraphics-fig.txt.

[MFont]

I'm sending the preflight pdf as an email attachment to you Michael. I don't know my way around github too well, and it's not setup on my work pc at the moment, so pardon the primitive method of distribution.

Some issues:

1. The chapter table of contents mindmap graphic contains RGB in the branches. I think I may have mentioned this a while ago, but apparently this is a problem with shadings in tikz. Some information is given in the answer here: http://tex.stackexchange.com/a/137031
2. The mindmap graphic also goes too far into the margins, and it would be cut off at printing. The trim marks indicate where the boundary of the printed page will be. (Edited to add...) I was a little hasty checking these; they appear to be within the trim marks, although they are living rather close to the edge. I'll check with someone to see how safe they are.
3. Some fonts aren't embedded. These are usually in the figures produced by R. There are many ways to fix this, both for inline graphics from R, and also using other programs for figures frozen with \includegraphics.
4. It's best to not use transparencies in the figures, weird things can happen at the printer. Transparencies should be flattened.
5. Imagemagick should work well for fixing some of these problems.

[friendly]

Marcus' Adobe preflight report, trimmed to about half size, is now in ChapmanHall/chapters_1-6_report-3.pdf. It does quite a good job of highlighting color issues in the first 6 chapters, with Adobe notes indicating the problems found.

(3) Font embedding: We fix this at the very end for the whole pdf. (4) Transparency: The most natural way to produce all the graphs with shaded regions in R is with transparency. Nothing we can do about this. I have no idea what 'flattening' means in terms of our figure files or how to flatten them. This should be an issue for the publisher/printer, not the authors.

[sciencegraph]

Should I proceed to fix some RGB to CMYK with ImageMagick?

[mattsigal]

All files listed in includegraphics-fig.txt have been addressed in this pull request. See the summary spreadsheet for details.

NOTE: Converting from RGB to CMYK does change shading rather drastically. I kept the original files (with -RGB appended to their filenames) so you can see them side by side. Probably won't make a huge difference in the long run in terms of reader use, but it is interesting to see.

Also: PNG was designed for the web, and cannot use CMYK. PNG files listed in the summary were necessarily converted to JPEG.

[friendly]

I didn't look at all the new vs. the -RGB images, but I didn't find any where CMYK made a huge difference. Were there any you remember where it changed shading 'drastically'?

[mattsigal]

Perhaps "drastically" is too strong, but here is a side by side comparison (CMYK on the left). When comparing across, the blues especially look strikingly different to me. As I said, probably not a big deal for the printing process/usefulness of the book - just something I thought I'd remark upon.

[friendly]

Now I see! This one is actually a very old graph, done with SAS. It uses fully saturated red and blue for the extreme residuals, the equivalent of gp=shading_Friendly in mosaic(). The blues do look different here, but I'm not going to worry about this because I don't think I can regenerate it.

I also see similar differences in col="blue" used in other graphs, for points and lines and fill. Not that big a deal, but they do look different.

Before I close this issue, could you say how you did the conversion? i.e., Acrobat/AI/Imagemagik on the image files? For R figures that got "frozen" via '\includegraphics{}, it might be useful to try re-generating the figure from the R source, but usinggrDevices::pdf.options(colormodel = "cmyk")`.

One or two tests should suffice to see if this makes a difference. E.g., the file ch06/R/ca-TV.R has the code producing ch06/fig/TV2-ca.pdf and ch06/fig/TV2-mosaic.pdf

[mattsigal]

Of course! It depended on the source file, but each conversion was done with an Adobe product: -PNG files were opened in Adobe Photoshop and converted to CYMK (using "Convert to Profile") -PDF files were opened in Adobe Acrobat Pro and converted to CYMK (using "Convert Colors") -Files that had a source AI file were output with CYMK encoding directly using Adobe Illustrator

I'll look at regenerating the files/indicating which could be done; my only hesitance on this front is that I'm not sure I can reknit a chapter to see what they look like in the compiled document?

[friendly]

Thanks. Let's regard PNG and AI conversions as done. It is only the PDF files that you converted with Adobe Acrobat Pro, vs. the alternatives that might be directly regenerated in R, with a CMYK setting for PDF output that I'm concerned with.

I did the quick test I suggested using a new source file, ch06/R/ca-TV-CMYK.R. It produces -CMYK versions of the two graphs of the TV2 data. Can you confirm (by visual comparison) that they are sufficiently close to the versions you did using Adobe Acrobat?

[mattsigal]

I just did side by side comparisons of both graphics (TV2-mosaic and TV2-ca) and there is no difference between the graphics produced directly by R and those that were converted manually by Acrobat. Here are the screenshots - with the R files designed in the filename at the top.

[friendly]

I ran an Adobe Acrobat Pro pre-flight report, now in book_preflight.pdf. A quick scan shows that two figures still get flagged as using RGB:

Fig 1.8, Fig 1.12

There may be others. I'm ignoring the problems with the tikz mindmaps for now. @mattsigal : please put this on your list of things to look at for the next revision. I also saw a chunk or two that were flagged, but I can't see why.

[mattsigal]

OK, I've made a note. This refers to: {fig:arrests0-star} {fig:nyt_512paths}

I found the cause for arrests0 - I converted the file in the fig/ directory, but the chunk actually refers to one in the fig/TorStar/ directory. Shouldn't be an issue to fix.

For the NYT graphic, that one wasn't on the list that I was working off of (from includegraphics-fig.txt). I can convert that no problem though for the next revision.

[friendly]

AFAICS, this issue can now be closed