search

pgf-tikz / pgf

A Portable Graphic Format for TeX
https://pgf-tikz.github.io/
1.13k stars 108 forks source link

external + "path fading" + xelatex -> broken #833

Closed letzfets closed 4 years ago

letzfets commented 4 years ago

see here: https://tex.stackexchange.com/questions/326756/tikz-externalize-breaks-path-fading The combination of the 3 above breaks the path fading command for all fadings, that are user defined, e.g. through \tikzfading

letzfets commented 4 years ago

As a workaround the \shade[] command works. Might do the job, as long as the fadings are not complicated.

hmenke commented 4 years ago

No MWE. What's wrong with using the issue template?

letzfets commented 4 years ago

Here we go:

Contents of file main.tex:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{fadings,external}
\tikzexternalize[prefix=figures/]
\begin{document}
\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
  \fill[blue!50, path fading=west]
  (7,-.1) -- (12,-.1) -- (12,-.2) -- (12.5,0) -- (12,.2) -- (12,.1) -- (7,.1) -- (7,-.1);
\end{tikzpicture}
\end{document}

I compiled with both xelatex -shell-escape main and with pdflatex -shell-escape main. Result is the same: the two faded arrows are showing up in the file figures/main-figure0.pdf, but not in main.pdf.

When I comment the line \tikzexternalize[prefix=figures/], everything is fine.

hmenke commented 4 years ago

Can't reproduce. Either your viewer is broken or this is a duplicate of #824.

letzfets commented 4 years ago

I checked. #824 works fine on my machine. No problem.

As viewers, I've used skim Version 1.5.6 (122) and preview Version 11.0 (999.4) both on my Mac - none of them work.

Other potentially relevant system information: pgf 3.1.5b pdfTeX 3.14159265-2.6-1.40.20 (TeX Live 2019) XeTeX 3.14159265-2.6-0.999991 (TeX Live 2019) macOS Catalina 10.15.3 (19D76)

I've also checked if it matters, wether I've got the Mac in dark scheme or light scheme mode (this had implications in other software before). But no.

What else do you need to reopen this as a bug. I'm willing to help - just got pretty frustrated yesterday about the fadings (spent too much time on it). Please let me know, what I can do for you to help you improve pgf!

Keep up the good work.

letzfets commented 4 years ago

I've viewed the main.pdf file now on my Android phone with Adobe Acrobat 20.0.1.11139. Both arrows show up there wonderfully. So keep closed here. This indicates, that it is a viewer issue.

hmenke commented 4 years ago

I think this is a duplicate of #824. Unfortunately, it's really unclear what is wrong with the fadings in XeTeX at the PDF level.

hmenke commented 4 years ago

Discussion on #824 revealed that this might not be the same problem. I'm happy to fix this but I need a reliable route through which I can reproduce this. Since I am on GNU/Linux x86_64 it can't involve Adobe Reader (or anything else that doesn't run on Linux).

Mo-Gul commented 4 years ago

For the record: Using Windows/MiKTeX, compiling with pdflatex and lualatex the above MWE works perfectly fine, both showing in SumatraPDF and Acrobat Reader.

letzfets commented 4 years ago

Discussion on #824 revealed that this might not be the same problem. I'm happy to fix this but I need a reliable route through which I can reproduce this. Since I am on GNU/Linux x86_64 it can't involve Adobe Reader (or anything else that doesn't run on Linux).

Skim and Preview are not Adobe either. ;-) Hmmm, How can I help?!? What viewer are you using? I might be able to install this one on my Mac and we'll take it from there?!?

hmenke commented 4 years ago

Skim and Preview don't run on GNU/Linux. I'm mostly using Evince and Zathura, but anything poppler- or mupdf-based should do. The pdf.js viewer which comes with Firefox is also a possibility.

letzfets commented 4 years ago

I was trying out the comment from @ilayn on #824 today (change from 50 to 100), but it had no effect on this one. I thought it might be helpful for further debugging, to provide you with the pdf's, where

main-figure0.pdf main.pdf main_no_subfolder.pdf

I ran a diff -a on those files and surprisingly the main.pdf and the main_no_subfolder.pdf give me an awful lots of hits (which might be due to a shift in the beginning of the document). My intuition tells me, that those files should ideally be exactly the same.

Hmm, the mystery continues! :wink:

hmenke commented 4 years ago

Since I don't know what I'm looking for and I have no way of reproducing this, I don't have any use for these PDFs. Also including a drawing is not the same as including a PDF with that drawing. There is a lot more stuff going on, like embedding of xref tables, etc.

hmenke commented 4 years ago

@letzfets Could you have a look at #841 and see whether you can reproduce it? I have a feeling that this is related.

hmenke commented 4 years ago

I have a hint here. The XeTeX driver is the only one that falls back on \includegraphics for \pgfimage (which is used to include externalized pictures). https://github.com/pgf-tikz/pgf/blob/357bc05926a057f7c1fcbc95877fc508723ac105/tex/generic/pgf/systemlayer/pgfsys-xetex.def#L31-L45

Here I have manually patched out the \includegraphics variant. @letzfets Can you try with this please?

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{fadings,external}
\tikzexternalize[prefix=figures/]

\makeatletter
\def\pgfsys@defineimage{% width, height, page number
    \PackageWarning{pgf}{Using modified \string\pgfsys@defineimage}%
    \ifx\pgf@imagewidth\pgfutil@empty\edef\pgf@imagewidth{1bp}\fi% width?
    \ifx\pgf@imageheight\pgfutil@empty\edef\pgf@imageheight{1bp}\fi% height?
    \ifx\pgf@imagepage\pgfutil@empty\else\edef\pgf@imagepage{ page \pgf@imagepage}\fi%
    \ifx\pgf@imageinterpolate\pgfutil@empty\else\edef\pgf@imageinterpolate{ /Interpolate\space\pgf@imageinterpolate}\fi%
    \ifx\pgf@imagemask\pgfutil@empty\else\xdef\pgf@imagemask{ /SMask @\pgf@imagemask}\fi%
    \edef\pgf@image{\noexpand\hbox to \pgf@imagewidth{\vbox to \pgf@imageheight{\vfil\special{pdf:image width \pgf@imagewidth\space height \pgf@imageheight\space\pgf@imagepage\space(\pgf@filename) <<\pgf@imageinterpolate\pgf@imagemask\space>>}}\hfil}}%
}
\makeatother

\begin{document}
\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
  \fill[blue!50, path fading=west]
  (7,-.1) -- (12,-.1) -- (12,-.2) -- (12.5,0) -- (12,.2) -- (12,.1) -- (7,.1) -- (7,-.1);
\end{tikzpicture}
\end{document}
hmenke commented 4 years ago

I have downloaded the main.pdf and main_no_subfolder.pdf you posted earlier and uncompressed both using

$ qpdf --qdf --object-streams=disable main.pdf main_un.pdf
$ qpdf --qdf --object-streams=disable main_no_subfolder.pdf main_no_subfolder_un.pdf

Then I deleted everything apart from the stream that renders the picture and diff-ed the result with git diff --no-index --word-diff main_un.pdf main_no_subfolder_un.pdf

--- a/main_un.pdf
+++ b/main_no_subfolder_un.pdf
@@ -1 +1 @@
 q 1 0 0 1 72 [--64.063-]{+720+} cm q 1 0 0 1 [--72 68.032-]{+76.712 -58.796+} cm q  0 G  0 g  0.3985 w  q  q  0.75 0 0.25 RG 0.75 0 0.25 rg 2.19131 0.0 0.0 0.15936 -54.99976 -7.96823 cm  /pgfsmask6 gs 0.4564 0.0 0.0 6.27516 25.10219 50.00197 cm  0.0 -1.98438 m  99.2134 -1.98438 l  99.2134 -3.96846 l  109.13472 0.0 l  99.2134 3.96846 l  99.2134 1.98438 l  0.0 1.98438 l  0.0 -1.98438 l  f  0 G 0 g Q  q  0.5 0.5 1 RG 0.5 0.5 1 rg 2.19131 0.0 0.0 0.15936 83.89897 -7.96823 cm  /pgfsmask6 gs 0.4564 0.0 0.0 6.27516 -38.29195 50.00197 cm  138.89876 -1.98438 m  238.11215 -1.98438 l  238.11215 -3.96846 l  248.03348 0.0 l  238.11215 3.96846 l  238.11215 1.98438 l  138.89876 1.98438 l  138.89876 -1.98438 l  f  0 G 0 g Q  Q  n  Q  Q {+BT /F1 9.9626 Tf 231.133 -630.635 Td[<0052>]TJ ET+} Q

The only things that are different are the shifts in the transformation and the presence of the text object which is in a different stream in main.pdf. I think this is most likely a bug in the viewer (which is also why I can't reproduce it) and you report it there.

letzfets commented 4 years ago

@letzfets Could you have a look at #841 and see whether you can reproduce it? I have a feeling that this is related.

I've replied in #841 , no I cannot reproduce it, sorry

letzfets commented 4 years ago

I have a hint here. The XeTeX driver is the only one that falls back on \includegraphics for \pgfimage (which is used to include externalized pictures).

https://github.com/pgf-tikz/pgf/blob/357bc05926a057f7c1fcbc95877fc508723ac105/tex/generic/pgf/systemlayer/pgfsys-xetex.def#L31-L45

Here I have manually patched out the \includegraphics variant. @letzfets Can you try with this please?

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{fadings,external}
\tikzexternalize[prefix=figures/]

\makeatletter
\def\pgfsys@defineimage{% width, height, page number
    \PackageWarning{pgf}{Using modified \string\pgfsys@defineimage}%
    \ifx\pgf@imagewidth\pgfutil@empty\edef\pgf@imagewidth{1bp}\fi% width?
    \ifx\pgf@imageheight\pgfutil@empty\edef\pgf@imageheight{1bp}\fi% height?
    \ifx\pgf@imagepage\pgfutil@empty\else\edef\pgf@imagepage{ page \pgf@imagepage}\fi%
    \ifx\pgf@imageinterpolate\pgfutil@empty\else\edef\pgf@imageinterpolate{ /Interpolate\space\pgf@imageinterpolate}\fi%
    \ifx\pgf@imagemask\pgfutil@empty\else\xdef\pgf@imagemask{ /SMask @\pgf@imagemask}\fi%
    \edef\pgf@image{\noexpand\hbox to \pgf@imagewidth{\vbox to \pgf@imageheight{\vfil\special{pdf:image width \pgf@imagewidth\space height \pgf@imageheight\space\pgf@imagepage\space(\pgf@filename) <<\pgf@imageinterpolate\pgf@imagemask\space>>}}\hfil}}%
}
\makeatother

\begin{document}
\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
  \fill[blue!50, path fading=west]
  (7,-.1) -- (12,-.1) -- (12,-.2) -- (12.5,0) -- (12,.2) -- (12,.1) -- (7,.1) -- (7,-.1);
\end{tikzpicture}
\end{document}

I've ran your patch both through xelatex and pdflatex. It does not solve the problem. Remember: in my case the problem shows up regardless of which of those two compilers I use.

Cheers

letzfets commented 4 years ago

I have downloaded the main.pdf and main_no_subfolder.pdf you posted earlier and uncompressed both using

$ qpdf --qdf --object-streams=disable main.pdf main_un.pdf
$ qpdf --qdf --object-streams=disable main_no_subfolder.pdf main_no_subfolder_un.pdf

Then I deleted everything apart from the stream that renders the picture and diff-ed the result with git diff --no-index --word-diff main_un.pdf main_no_subfolder_un.pdf

--- a/main_un.pdf
+++ b/main_no_subfolder_un.pdf
@@ -1 +1 @@
 q 1 0 0 1 72 [--64.063-]{+720+} cm q 1 0 0 1 [--72 68.032-]{+76.712 -58.796+} cm q  0 G  0 g  0.3985 w  q  q  0.75 0 0.25 RG 0.75 0 0.25 rg 2.19131 0.0 0.0 0.15936 -54.99976 -7.96823 cm  /pgfsmask6 gs 0.4564 0.0 0.0 6.27516 25.10219 50.00197 cm  0.0 -1.98438 m  99.2134 -1.98438 l  99.2134 -3.96846 l  109.13472 0.0 l  99.2134 3.96846 l  99.2134 1.98438 l  0.0 1.98438 l  0.0 -1.98438 l  f  0 G 0 g Q  q  0.5 0.5 1 RG 0.5 0.5 1 rg 2.19131 0.0 0.0 0.15936 83.89897 -7.96823 cm  /pgfsmask6 gs 0.4564 0.0 0.0 6.27516 -38.29195 50.00197 cm  138.89876 -1.98438 m  238.11215 -1.98438 l  238.11215 -3.96846 l  248.03348 0.0 l  238.11215 3.96846 l  238.11215 1.98438 l  138.89876 1.98438 l  138.89876 -1.98438 l  f  0 G 0 g Q  Q  n  Q  Q {+BT /F1 9.9626 Tf 231.133 -630.635 Td[<0052>]TJ ET+} Q

The only things that are different are the shifts in the transformation and the presence of the text object which is in a different stream in main.pdf. I think this is most likely a bug in the viewer (which is also why I can't reproduce it) and you report it there.

Well, it might very well be, that both viewers (skim and preview). I've also installed Firefox now, and it's the same result there. Hmm?!?

hmenke commented 4 years ago

I don't know what exactly you are doing but both arrows show up correctly for me in Firefox 74.0, so this might not even be a viewer issue but some other issue with your specific setup.

test

Sorry, but if you can't come up with a way for me to reproduce this (that doesn't involve me buying a €2000 MacBook), then I will have to close this. I'll also ask in the TeX.SX whether anyone can reproduce.

hanche commented 4 years ago

I can reproduce it on my mac (recent macbook pro, latest macos, TL 2020 pretest).

The resulting pdf has no visible arrows when viewed in skim, preview, or pdfpen pro. However, with mupdf, the arrows are present. So it seems like it could be a viewer issue. I get the same result when compiling on a gnu/linux machine and viewing the result on the mac.

letzfets commented 4 years ago

Then we have at least 3 computers on this planet, that got this problem. Note, that the original MWE, that I provided was from 2016 here https://tex.stackexchange.com/questions/326756/tikz-externalize-breaks-path-fading I can try to have a look at my girlfriends MacBook Pro (a few years older hardware and a few MacOS versions older than mine). Personally I have worked around the issue in my document with the \shade command. But I see it as a sporty challenge to find the root of this issue.

hmenke commented 4 years ago

Reproducing this on more macOS machines doesn't make sense at this point because we already know that Apple's PDF engine fails to render this correctly. Instead you should try to reproduce this on computers that are not running macOS, so you can be sure that the viewer definitely cannot fall back onto Apple's broken PDF engine (which is what might happen for your Firefox). Furthermore you should also report the bug to Apple and hear what they have to say about it.

u-fischer commented 4 years ago

@letzfets @hanche what happens if the shading is in an external and in an internal picture?

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{fadings,external}

\begin{document}
\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
\end{tikzpicture}

\tikzexternalize[]
\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
\end{tikzpicture}
\end{document}
hanche commented 4 years ago

@u-fischer That doesn't work because \tikzexternalize has to be called in the preamble. It ends up calling \nofiles … and perhaps there are other reasons.

u-fischer commented 4 years ago

@hanche yes sorry it worked for me (or did what I expected) but probably only because the graphic existed. Can you try this:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{fadings,external}
\tikzexternalize[]
\begin{document}
\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
\end{tikzpicture}

\tikzexternaldisable

\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
\end{tikzpicture}
\end{document}
hanche commented 4 years ago

In that case, only the second arrow shows up. (To be certain, I changed the colour on the second one.)

u-fischer commented 4 years ago

No don't change the color! The shadings must be identical.

hanche commented 4 years ago

Only one showed up when they were identical, too.

u-fischer commented 4 years ago

Yes, I just checked the numbers the shading resources have different names so this can't work. I need to think about it, there must be a way to get the resource in the page dictionary at least for test ;-)

u-fischer commented 4 years ago

@hanche Well let's try this (I'm not sure about the number I copied them out of my pdf, but it could be that your pdf actually creates others):

\documentclass{article}
\usepackage{l3pdf}
\ExplSyntaxOn
\pdf_uncompress:
\ExplSyntaxOff
\usepackage{tikz}
\usetikzlibrary{fadings,external}
\tikzexternalize[]
\begin{document}
\makeatletter \pgf@sys@addpdfresource@extgs@plain{%
 /pgfsmask28 <<
            /SMask <<
                 /S /Luminosity
                 /G 35 0 R
                  >>
             >>
 /pgfsmask35 <<
           /SMask <<
             /S /Luminosity
             /G 36 0 R
                 >>
             >>} 
\begin{tikzpicture}[scale=0.7]
  \fill[purple, path fading=west]
  (0,-.1) -- (5,-.1) -- (5,-.2) -- (5.5,0) -- (5,.2) -- (5,.1) -- (0,.1) -- (0,-.1);
\end{tikzpicture}
\end{document}
hanche commented 4 years ago

That made a blank page.

u-fischer commented 4 years ago

Can you sent me the pdf? Then I can check the numbers.

hanche commented 4 years ago

Note to others: We moved this discussion to the TeX stackexchange chat. Not much useful information emerged, unfortunately.

Mo-Gul commented 4 years ago

Maybe others want to follow too, you could paste the link to the chat here.

hmenke commented 4 years ago

We also have a mailing list at pgf-tikz@tug.org where you can have extended discussions (sign up here https://tug.org/mailman/listinfo/pgf-tikz).

hanche commented 4 years ago

Nothing much to see there (on the tex.se chat) really. Ulrike wanted to test some hypothesis, adding dictionaries to the pdf to see what happens. Since she does not have a Mac, I ran her code and checked the result. The hypothesis did not pan out, so that's the end of the story. But if you really are curious, start here:

https://chat.stackexchange.com/transcript/message/53887809#53887809

hmenke commented 4 years ago

I'm pretty convinced by now that this an issue with Apple's PDF rendering engine. So I'm closing this again.