Memoize doesn't work together with mylatexformat

[HGSqm]

In the past I used the package mylatexformat to speed up compilation of documents with a lot of packages in the preamble. This creates a kind of precompiled preamble which is just loaded instead of recompiling everything at each run. Now that I discovered memoize I tried to combine both, unfortunately without success. The tikzpicture environments get extracted, but no PDF file is created.

MWE (file "t1-memoize.tex"):

%&"t1-memoize-preamble"

\documentclass{article}

\usepackage{tikz}
\usepackage{memoize}

\csname endofdump\endcsname
% precompiled format ends here

\begin{document}

\begin{tikzpicture}
  \draw (0,0) -- (4,1);
\end{tikzpicture}

\end{document}

The precompiled format is created with the command

etex -initialize -save-size=20000 -stack-size=20000 -jobname="t1-memoize-preamble" "&pdflatex" mylatexformat.ltx "t1-memoize.tex"

This creates a file t1-memoize-preamble.fmt which is then loaded when running pdflatex on the main document.

pdflatex t1-memoize.tex

But this just produces the message

Package memoize Warning: The compilation produced 1 new extern on input line 17

on every run. A file t1-memoize.mmz is created with the following content:

\mmzPrefix {t1-memoize-preamble.memo.dir/}
\mmzUsedCMemo {t1-memoize-preamble.memo.dir/FFD5A3FD2DAAE9672BFD187082C04266.memo}
\mmzUsedCCMemo {t1-memoize-preamble.memo.dir/FFD5A3FD2DAAE9672BFD187082C04266-E778DCCCB8AAB0BBD3F6CFEEFD2421F8.memo}
\mmzNewCMemo {t1-memoize-preamble.memo.dir/FFD5A3FD2DAAE9672BFD187082C04266.memo}
\mmzNewCCMemo {t1-memoize-preamble.memo.dir/FFD5A3FD2DAAE9672BFD187082C04266-E778DCCCB8AAB0BBD3F6CFEEFD2421F8.memo}
\mmzNewExtern {t1-memoize-preamble.memo.dir/FFD5A3FD2DAAE9672BFD187082C04266-E778DCCCB8AAB0BBD3F6CFEEFD2421F8.pdf}{1}{258.75095pt}{173.39272pt}
\endinput 

The file t1-memoize.mmz.log is empty. The directory t1-memoize-preamble.memo.dir is there with two files with extension .memo.

My test environment is a fresh installation of MiKTeX portable with recent updates. I use a recent strawberry perl, but I think that is ok as memoize works fine when the precompiled format is omitted.

My impression is that creating the format file fixes something for memoize which points to a wrong file/directory. Moving \usepackage{memoize} a bit down so it is not included in the format file makes everything work. But the manual says memoize likes being loaded early.

I tried creating the format file with the same basename as the main document (t1-memoize), but that didn't change anything.

[sasozivanovic]

By default, Memoize performs the extern extraction when the package is loaded, so things can't work out of the box with \usepackage{memoize} baked into the format (i.e. the precompiled preamble).

My suggestion: \usepackage[extract=no]{memoize} in the precompiled part of the preamble, followed by \mmzset{extract} (or \mmzset{extract=perl} etc.) in the part of the preamble which is not precompiled — but necessarily before \begin{document}. I'm not familiar with mylatexformat, but I hope there is a way to mark only a part of the preamble to be precompiled.

Please report on the success, and feel free to ask further questions.

[HGSqm]

Thank you very much for the explanation and suggestion, this works perfectly.

Indeed mylatexformat provides a mechanism to end the precompilation. Everything after \csname endofdump\endcsname (or just \endofdump) is compiled every time. The example below now works.

%&"t1-memoize-preamble"

\documentclass{article}

\usepackage{tikz}
\usepackage[extract=no]{memoize}

\csname endofdump\endcsname
% precompiled format ends here

\mmzset{extract}

\begin{document}

\begin{tikzpicture}
  \draw (0,0) -- (4,1);
\end{tikzpicture}

\begin{tikzpicture}
  \draw (0,0) -- (-4,1);
\end{tikzpicture}

\end{document}

In fact now that I know what to look for I see that you describe the case of a user format in section 5.5 of the manual. Unfortunately I did't realize that before.

[sasozivanovic]

I'm glad it does, and that I could help!

In fact now that I know what to look for I see that you describe the case of a user format in section 5.5 of the manual.

Yeah, I had a vague feeling someone might try do do this. ;-)

[amonakov]

Hi! We discussed this earlier in issue #6.

[sasozivanovic]

Indeed. I forgot ;-)

[HGSqm]

Maybe you could mention this in the "Known issues" section of the manual. I guess that's the first place where users would look at if something doesn't work as expected.