per overlay only works in beamer mode

[cfr42]

If overlay specifications are used in handout mode, per overlay has no effect because the dummy \only which gets \mmzBeamerOverlays into the cc-memo cannot tell Beamer to create any additional slides in handout mode.

If you only ever produce a single slide from a frame in handout mode, there's no problem, but if you produce multiple slides, per overlay causes the externs to be created, but they are never used. So something like the following is needed,

    \mmzset{% 
        per overlay/.style={%  t. 47 
          /mmz/context={overlay=\csname beamer@overlaynumber\endcsname},
          /mmz/at begin memoization={%
            \xtoksapp\mmzCMemo{%
              \gdef\noexpand\mmzBeamerOverlays{\beamer@overlaynumber}%
            }%
            \gtoksapp\mmzCCMemo{%
              \only<0| handout:\mmzBeamerOverlays>{}% <= create multiple slides in handout mode
            }%
          },
        }
      }

However, as it stands, this will obviously stop per overlay working in beamer mode, too. Moreover, if you produce both forms from a single source, the externs of one mode may silently replace (some of) the other's.

So you actually need to make it sensitive to whether you're producing in handout or beamer mode and to use a different prefix. I already had a function to add to the prefix Memoize uses, which I wrote after dir changed to prefix. You won't like it because it uses expl3, so let's just say that \cfg_memoize_addtoprefix:n appends -#1 to the current prefix or sets the prefix to <memo directory>/#1 if there isn't one.

Apologies for the expl3:

\hook_gput_code:nnn { class/beamer/after } { memoize ~ cfg }
{% don't know if this is the right hook: I'm actually adding it after loading a custom package which defines overlay keys for tikz/forest
  \hook_gput_code:nnn { begindocument/before } { memoize ~ cfg }
  {% not sure this is the right hook either
    \mode<handout>
    {
      \cfg_memoize_addtoprefix:n { h }
      \mmzset{
        per ~ overlay/.style={% docs t. 47 
          /mmz/context={overlay=\csname beamer@overlaynumber\endcsname},
          /mmz/at ~ begin ~ memoization={
            \xtoksapp\mmzCMemo{
              \gdef\noexpand\mmzBeamerOverlays{\beamer@overlaynumber}
            }
            \gtoksapp\mmzCCMemo{
              \only<0| ~ handout:\mmzBeamerOverlays>{}
            }
          },
        }
      }
    }
    \mode<beamer>
    {
      \cfg_memoize_addtoprefix:n { b }
    }
  }
}

This produces e.g. <original prefix>-h-<string>-<string>.memo etc.

MWE:

\RequirePackage{memoize}
\ExplSyntaxOn
\makeatletter
\tl_new:N \l_cfg_memoize_dir_tl
\tl_set:Nn \l_cfg_memoize_dir_tl { memos }
\tl_new:N \l_cfg_memoize_fileprefix_tl
\tl_set_eq:NN \l_cfg_memoize_fileprefix_tl \c_sys_jobname_str
\cs_new_protected_nopar:Nn \cfg_memoize_setprefix:n
{
  \mmzset {
    prefix = { \l_cfg_memoize_dir_tl / #1 - }
  }
  \tl_set:Nn \l_cfg_memoize_fileprefix_tl { #1 }
}
\cs_generate_variant:Nn \cfg_memoize_setprefix:n { V }
\cs_new_protected_nopar:Nn \cfg_memoize_addtoprefix:n
{
  \tl_if_empty:NTF \l_cfg_memoize_fileprefix_tl
  {
    \cfg_memoize_setprefix:n { #1 }
  }{
    \tl_put_right:Nn \l_cfg_memoize_fileprefix_tl { - #1 }
    \cfg_memoize_setprefix:V \l_cfg_memoize_fileprefix_tl
  }
}
\cs_generate_variant:Nn \cfg_memoize_addtoprefix:n { V }
\hook_gput_code:nnn { begindocument } { . memoize ~ cfg }
{
  \ProvideDocumentCommand \tikzpicturedependsonfile { m }
  {
    \tl_gput_right:Nn \mmzContextExtra {#1}
  }
}
\hook_gput_code:nnn { class/beamer/after } { memoize ~ cfg }
{% don't know if this is the right hook: I'm actually adding it after loading a custom package which defines overlay keys for tikz/forest
  \hook_gput_code:nnn { begindocument/before } { memoize ~ cfg }
  {% not sure this is the right hook either
    \mode<handout>
    {
      \cfg_memoize_addtoprefix:n { h }
      \mmzset{
        per ~ overlay/.style={% docs t. 47 
          /mmz/context={overlay=\csname beamer@overlaynumber\endcsname},
          /mmz/at ~ begin ~ memoization={
            \xtoksapp\mmzCMemo{
              \gdef\noexpand\mmzBeamerOverlays{\beamer@overlaynumber}
            }
            \gtoksapp\mmzCCMemo{
              \only<0| ~ handout:\mmzBeamerOverlays>{}
            }
          },
        }
      }
    }
    \mode<beamer>
    {
      \cfg_memoize_addtoprefix:n { b }
    }
  }
}
\makeatother
\ExplSyntaxOff
% \documentclass{beamer}
\documentclass[handout]{beamer}

\usepackage{tikz}
\tikzset{% common overlay specs for beamer and for compatibility cases
  visible on/.style={alt=#1{}{invisible}},
  opaque on/.style={alt=#1{}{transparent}},
  % set up for transitions using tikz with beamer overlays - developed by Daniel (http://tex.stackexchange.com/a/55849/) and, in earlier form, by Matthew Leingang (http://tex.stackexchange.com/a/6155/) and modified for this use, I think by Qrrbrbirlbel (http://tex.stackexchange.com/a/112471/)
  invisible/.style={opacity=0,text opacity=0},
  alt/.code args={<#1>#2#3}{%
    \alt<#1>{\pgfkeysalso{#2}}{\pgfkeysalso{#3}} % \pgfkeysalso doesn't change the path
  },
  transparent/.style={opacity=0.1,text opacity=0.1},
}
\begin{document}
\begin{frame}<1-| handout:1-2>{A Frame}
  \begin{tikzpicture}[/mmz/per overlay]
    \node at (0,0) {origin};
    \node [visible on=<2-| handout:1-2>] at (1,0) {right};
    \node [visible on=<3-| handout:1-2>] at (-1,0) {left};
    \node [visible on=<4-| handout:2>] at (0,1) {above};
    \node [visible on=<5-| handout:2>] at (0,-1) {below};
  \end{tikzpicture}
\end{frame}
\end{document}

This produces 2 externs memos/<filename>-h-<memo-id>-<context-id>.pdf. Switching to beamer mode produces 5 memos/<filename>-b-<memo-id>-<context-id>.pdf, Obviously the specifics are highly idiosyncratic, but it does get per overlay to work as expected.

[sasozivanovic]

Sorry I haven't been able to look into this yet, it's been busy on my side. Hope I can get to it soon.

[cfr42]

No worries. Thanks for the heads up. My fix works fine for me anyway ;).

[sasozivanovic]

Thank you for raising this issue! Indeed, Beamer support seems only half-baked at the moment!

A agree with your analysis that \only is used incorrectly in the cc-memo, but I would propose a different solution: simply prefix its <mode specification> with all:, so that the definition of per overlay would read as below (based on the real code from the .sty rather than on the "first attempt" in the docs). This would make it work not only in the default beamer mode, and in the handout mode, but in any mode. Do you see any disadvantages of the approach?

  \mmzset{per overlay/.style={
    /mmz/context={%
      overlay=\csname beamer@overlaynumber\endcsname,
      pauses=\ifmemoizing
               \mmzBeamerPauses
             \else
               \expandafter\the\csname c@beamerpauses\endcsname
             \fi
    },
    /mmz/at begin memoization={%
      \xdef\mmzBeamerPauses{\the\c@beamerpauses}%
      \xtoksapp\mmzCMemo{%
        \noexpand\mmzSetBeamerOverlays{\mmzBeamerPauses}{\beamer@overlaynumber}}%
      \gtoksapp\mmzCCMemo{%
        \only<all:\mmzBeamerOverlays>{}}%
    },
    /mmz/at end memoization={%
      \xtoksapp\mmzCCMemo{%
        \noexpand\setcounter{beamerpauses}{\the\c@beamerpauses}}%
    },
    /mmz/per overlay/.code={},
  }}

Regarding "mode-mixing", e.g. compiling both the presentation and the handout from the same source, I first thought that a simple csname meaning to context=beamer@currentmode will suffice. This would produce different externs for different modes. But this won't work because different modes might produce a different number of overlays (for some frame), and the number of overlays is baked into the c-memo, which would be the same for all modes.

So your solution is certainly the right one: by changing the prefix wrt the mode, we get a separate c-memo for each mode, effectively pretending that we're compiling different sources, so everything should work out of the box.

You're right, the expl3 code scares me a bit ;-) and I won't pretend I fully understand it. My question is, is there anything that the simple approach below cannot handle?

\mmzset{
  beamer mode to prefix/.style={
    prefix=\mmz@prefix@dir\mmz@prefix@name\beamer@currentmode_mode.
  },
}

The idea is for the user to write beamer mode to prefix either into the document preamble, or for particular pictures. The document-wide and the \mmzset-based usage work fine for me; using this (and prefix in general) with \mmznext will require that I fix some code, but I believe it is important that the approach can work on per-picture basis — some pictures are different in different modes, and other pictures are not. (This is just like with per overlay, where pictures might or might not depend on the overlay number.)

[cfr42]

It seems to work in very minimal testing. I'll try to test a bit more.

How safe will it be to redefine

  beamer mode to prefix/.style={
    prefix=\mmz@prefix@dir\mmz@prefix@name\beamer@currentmode_mode.
  },

?

I think it would be safer to use

        \only<presentation:\mmzBeamerOverlays>{}}%

rather than

        \only<all:\mmzBeamerOverlays>{}}%

Per picture sounds fine. As long as I can set it globally. True, this is slightly wasteful - I compile some pictures twice when once would have been sufficient. But it is significantly more reliable not to rely on my figuring out which ones require special treatment, especially when I'm just updating existing presentations.

As it stands, memoize can't cope with article mode, but it seems to be fairly easy to add support? I tested https://github.com/cfr42/memoize-mod/commit/d6f2f9419aa71e1d9be45662f1465739b0228772 and could get mode-aware externs for article, beamer, handout and trans. (Am I missing one? I thought there was a notes one, but that option just seemed to reproduce the beamer.)

I haven't tested with anything non-minimal yet.

[Note that I used \ifdef but my understanding of etoolbox is virtually nil.]

[sasozivanovic]

It seems to work in very minimal testing. I'll try to test a bit more.

Great, thanks!

How safe will it be to redefine

  beamer mode to prefix/.style={
    prefix=\mmz@prefix@dir\mmz@prefix@name\beamer@currentmode_mode.
  },

?

Completely safe, I believe. You could also define your own style, or use prefix directly. (I assume that you would want some other form of the prefix, like in your original code? That should be fine.)

I think it would be safer to use

        \only<presentation:\mmzBeamerOverlays>{}}%

rather than

        \only<all:\mmzBeamerOverlays>{}}%

A good idea. While I have the impression that the two might be equivalent (as the article mode shouldn't generate overlays anyway), there might be some corner cases where presentation: is safer. I'll think about this some more.

Per picture sounds fine. As long as I can set it globally.

Definitely. You could even put it in memoize.cfg, e.g. with \mmzset{beamer mode to prefix/.try}, to be loaded for every beamer document.

True, this is slightly wasteful - I compile some pictures twice when once would have been sufficient. But it is significantly more reliable not to rely on my figuring out which ones require special treatment, especially when I'm just updating existing presentations.

And this is why I don't dare to make it the default, even if I will probably make it a default for my own presentations as well.

As it stands, memoize can't cope with article mode, but it seems to be fairly easy to add support? I tested cfr42@d6f2f94 and could get mode-aware externs for article, beamer, handout and trans.

Thanks, I have learned more about the article mode now, checking for beamerarticle is certainly the way to go.

(Am I missing one? I thought there was a notes one, but that option just seemed to reproduce the beamer.)

The notes mode (called second, for the second screen) seems to a fake mode. One can use second: in a mode specification, but under the hood, Beamer still runs in the beamer mode. I'm trying to figure out whether there are situations where Memoize should distinguish the two modes (beamer and second). Perhaps it would be best to leave things as they are and see if anyone runs into trouble?

I haven't tested with anything non-minimal yet.

I have just pushed the Beamer mode related update to GitHub (24060d97d4050c4994579ffb8331bc3cdcd104a7). I tested it with some of my old presentations, but only to confirm that it does no harm in beamer mode. I never wrote a real document using handout or article mode to test it against.

[cfr42]

Sorry, I somehow missed your latest response. I've been experimenting a bit with this. I don't usually use article mode but I have a series of documents for workshops I was teaching I can test on. So far, I'm not doing very well.

What does

runsystem(memoize-extract.pl  -F latex slides )...disabled

mean? The project uses a Makefile and has no problem when article mode is active, but it refuses to cooperate when beamer is in effect. Compiling the file directly works better, but I can't figure out why it should be different.

I have just pushed the Beamer mode related update to GitHub (https://github.com/sasozivanovic/memoize/commit/24060d97d4050c4994579ffb8331bc3cdcd104a7).

If you do it like this, I think you have to document that users can't use beamer mode to prefix in .cfg files directly? I was thinking what you could do is to define the key so it adds code to one of the begindocument hooks. This way it would always work as intended, whether it is used in the cfg file, in the preamble or in the body of the document. But maybe you just want to document that users can put it into a hook if they use it earlier?

There are some odd effects if the mode isn't added to the prefix by default for article mode because there may be lots of differences between pictures as a result of that context rather than due to anything the user has consciously changed. These differences needn't be a matter of overlays in the non-article version. In the example I have, the picture in article mode uses monochrome and serif, whereas the beamer one uses sans and colour. Also, any picture which uses, say, \textwidth or dimensions which depend on font size may turn out differently depending on the order in which the user compiles in different modes.

I wonder if it would be worth thinking about whether the mode should be added to the prefix by default specifically in article mode?

Off-topic: each time you edit an .edtx file, you have to regenerate the .dtx and .sty to test - is that correct?

[cfr42]

I finally figured out all-runtimes is the target I want to get just the .stys. I expected runtimes to work, but it always says there is nothing to do.

However, I still get failure at extraction specifically in beamer mode in my non-minimal case. Can I get more information about what's going wrong somehow?

\mmz@temptoks=\toks22
\mmz@box=\box52
\mmz@out=\write3
\mmz@mdfive@source=\toks23
\mmz@exec@source=\toks24
\mmzContext=\toks25
\mmzContextExtra=\toks26
\mmzCMemo=\toks27
\mmzCCMemo=\toks28
\mmz@seq=\count192
\mmz@ccmemo@resources=\toks29
\mmz@tbe@box=\box53
\mmzExternPages=\count193
\mmzExtraPages=\count194
\openout0 = `slides.mmz.log'.

runsystem(memoize-extract.pl  -F latex slides )...disabled.

(./slides.mmz.log)

! Package memoize Error: Extraction of externs from document "slides.pdf" using
method "perl" was unsuccessful.

See the memoize package documentation for explanation.
Type  H <return>  for immediate help.
...

l.1572 }

?
! Emergency stop.
...

l.1572 }

The extraction script "memoize-extract.pl" wasn't executed or didn't finish exe
cution properly.

The log file it refers to is invariably empty. It's not a general extraction problem because the article mode goes fine. It produces two PDFs in article mode without any trouble. But it balks in beamer.

Some incompatibility with make. If I compile the file directly, it was working. So I can't find the problem that way.

[cfr42]

I never wrote a real document using handout or article mode to test it against.

Would you like one?

[sasozivanovic]

What does

runsystem(memoize-extract.pl  -F latex slides )...disabled

mean? The project uses a Makefile and has no problem when article mode is active, but it refuses to cooperate when beamer is in effect. Compiling the file directly works better, but I can't figure out why it should be different.

The message means that memoize-extract.pl is not among the shell-escape commands. However, why the problem would only arise in the beamer mode is beyond me. I can only conclude that somehow, the Makefile executes the compilation in a different way for the two modes. Could you post the Makefile?

I have just pushed the Beamer mode related update to GitHub (24060d9).

If you do it like this, I think you have to document that users can't use beamer mode to prefix in .cfg files directly? I was thinking what you could do is to define the key so it adds code to one of the begindocument hooks. This way it would always work as intended, whether it is used in the cfg file, in the preamble or in the body of the document. But maybe you just want to document that users can put it into a hook if they use it earlier?

The issue is that beamer mode to prefix is only defined when beamer/beamerarticle is loaded. Therefore, when it is called from the config file, the execution in non-beamer documents has to be prevented somehow. I simply find beamer mode to prefix/.try a particularily concise way to do this. \@ifclass/packageloaded etc. would would just as well. But anyhow, I agree, this should be mentioned in the documentation — unless it is made the default, as you suggest below.

There are some odd effects if the mode isn't added to the prefix by default for article mode because there may be lots of differences between pictures as a result of that context rather than due to anything the user has consciously changed. These differences needn't be a matter of overlays in the non-article version. In the example I have, the picture in article mode uses monochrome and serif, whereas the beamer one uses sans and colour. Also, any picture which uses, say, \textwidth or dimensions which depend on font size may turn out differently depending on the order in which the user compiles in different modes.

I wonder if it would be worth thinking about whether the mode should be added to the prefix by default specifically in article mode?

This is a very good point. The problem is only that it is hard to undo the effect of beamer mode to prefix, when one doesn't want it. So I have to think about this.

Off-topic: each time you edit an .edtx file, you have to regenerate the .dtx and .sty to test - is that correct?

Yes.

I finally figured out all-runtimes is the target I want to get just the .stys. I expected runtimes to work, but it always says there is nothing to do.

Yeah, runtimes produces only Memoize runtimes; all-runtimes includes Advice and CollArgs. Having several packages in the same repo makes things more complicated ...

I never wrote a real document using handout or article mode to test it against.

Would you like one?

Yes, please!

[cfr42]

The message means that memoize-extract.pl is not among the shell-escape commands. However, why the problem would only arise in the beamer mode is beyond me. I can only conclude that somehow, the Makefile executes the compilation in a different way for the two modes. Could you post the Makefile?

Sorry. This took longer than it should because I know next to nothing about Makefiles.

I never wrote a real document using handout or article mode to test it against.

Would you like one?

Yes, please!

https://github.com/cfr42/latex-1-2

I think it should build on any system with TeX Live and a GNU tool chain, but I'm not absolutely certain about this. If so, it will create some broken symbolic links (though it may also remove them as it has a tendency to destroy things after making them, which I'm sure make didn't used to).

The Makefile invokes pdfLaTeX with --shell-escape and will attempt to create symbolic links to some files in TEXMFDIST. (Also TEXMFHOME, but that's set to empty.) However, it doesn't look to see if they exist or do anything with them.

No attempt has been made in this version to transition the project to Memoize, which is what I was trying to do. However, it does now build with the standard external library (albeit I had to exclude more pictures).

There are 15 core PDFs using 5 source files and 3 wrappers. The main sources are all called training.tex and the wrappers are handouts.tex, tutornotes.tex and slides.tex. The handouts and tutornotes use article mode. (handout mode isn't used here.)

Do you want one for handout mode, too? I don't have anything I can think of which uses both that and article, but I have plenty of examples which use beamer and handout.

[sasozivanovic]

Wow, this is one hell of a test file! Thank you!!!

It compiled fine once I adjusted the PATH at the top of the Makefile. I also skipped the font-sampler file, as it was trying to load fonts I don't have installed.

Hoping to find the time to play with this soon.

[cfr42]

Wow, this is one hell of a test file! Thank you!!!

It's the only real example I have which uses article mode. I use handout mode all the time, but not article.

It compiled fine once I adjusted the PATH at the top of the Makefile.

Ah! Sorry, yes.

I also skipped the font-sampler file, as it was trying to load fonts I don't have installed.

Hmm. Irrelevant for current purposes anyway. I should probably separate it out somehow.

Thanks for the feedback!

[cfr42]

[Edit: now at least minimally overlay-aware, but there is not actually much to do once it works. However, I've not actually looked at the output. ;)]

Compiles, but I haven't done anything to enable multiple overlays:

https://github.com/cfr42/latex-1-2/tree/memoize

May also build without needing to edit PATH or disable the fonts handouts ... or may not. Since I don't know what I'm doing, the latter is likely to the point of near certainty.

[cfr42]

If you would like an real case for handout mode, may I email you something? I don't have anything intended for public use, but if I take all of the custom stuff out, it makes it hard to keep the case 'real' and I'm also worried about inadvertently leaving something in which I shouldn't.

[cfr42]

The modified code I checked out (sorry - I know that's not git terminology: should it be cloned?) from GitHub reports externs used incorrectly in some cases. If you run the code from https://github.com/sasozivanovic/memoize/issues/26#issue-2335483579 (with slight changes to mmz settings: prefix=memos/,include context in ccmemo), .mmz reports using a non-existent extern:

\mmzPrefix {memos/}
\mmzUsedCMemo {memos/C6285CB0A79204B05AC8C338675603CE.memo}
\mmzUsedExtern {memos/C6285CB0A79204B05AC8C338675603CE-DFCE5067A8952F93E51E29446EB6D3E2-1.pdf}
\mmzUsedCCMemo {memos/C6285CB0A79204B05AC8C338675603CE-DFCE5067A8952F93E51E29446EB6D3E2.memo}
\mmzUsedCMemo {memos/3E9506E82B16BC2B8D51514961456FDF.memo}
\mmzNewCMemo {memos/3E9506E82B16BC2B8D51514961456FDF.memo}
\mmzNewCCMemo {memos/3E9506E82B16BC2B8D51514961456FDF-A309C642D1D944E3F0F132AEE1328602.memo}
\mmzNewExtern {memos/3E9506E82B16BC2B8D51514961456FDF-A309C642D1D944E3F0F132AEE1328602.pdf}{1}{220.86224pt}{161.81287pt}
\endinput

whereas

$ ls memos/
3E9506E82B16BC2B8D51514961456FDF-A309C642D1D944E3F0F132AEE1328602.memo
3E9506E82B16BC2B8D51514961456FDF-A309C642D1D944E3F0F132AEE1328602.pdf
3E9506E82B16BC2B8D51514961456FDF.memo
76D99DB4F7CD2A45CF1F310A7D391385-A309C642D1D944E3F0F132AEE1328602.memo
76D99DB4F7CD2A45CF1F310A7D391385-A309C642D1D944E3F0F132AEE1328602.pdf
76D99DB4F7CD2A45CF1F310A7D391385.memo
C6285CB0A79204B05AC8C338675603CE-DFCE5067A8952F93E51E29446EB6D3E2.memo
C6285CB0A79204B05AC8C338675603CE-DFCE5067A8952F93E51E29446EB6D3E2.pdf
C6285CB0A79204B05AC8C338675603CE.memo

This doesn't happen with the current CTAN version. (I'm assuming this is just minor housekeeping somewhere.)

[cfr42]

Ref.: https://tex.stackexchange.com/a/720529/

This answer reflects another way of using handout mode. It doesn't involve memoize or, even, a picture, but I thought it might be useful to test something like this, too.