TikZ externalise causes XSIM to produce lots of extra files

[zoqaeski]

I've got a document with exercises and lots of images which I'm externalising with TikZ to speed up compilation times. Due to some conflict between how externalize works and XSIM, my images directory is getting filled up with multiple copies of all the exercises—one set of all of them for each externalised image.

I've attached a working example to demonstrate; it's more of a maximal working example as I just made a subset of the aforementioned document.

The TikZ external system call expects a directory called tikz to exist in the same folder as the github-xsim-bug.tex, and to compile it you'll need to call lualatex once, followed by make -f github-xsim-bug.makefile, followed by a second lualatex invocation. The main file does compile with plain pdflatex, but the externalisation itself uses luatex. Once it's all compiled, you should get the main document, a set of images in tikz/, and a set of every exercise/solution pair for every image.

The TikZ external/mode=list and make is easier on my system's memory—and much quicker to compile—but commenting out line 43 will revert to the usual forking method using -shell-escape or -write18. Both methods still cause XSIM to spit out lots of additional files that are never actually used.

github-xsim-bug.txt (Github wouldn't let me upload *.tex so I renamed it *.txt.)

[cgnieder]

Please – can you reduce this to a MWE (at max one or two exercises, no packages other than the ones needed to recreate the core issue)?

I'm not able to see any problems with the following example:

\documentclass{article}

\usepackage{tikz}
\usetikzlibrary{external}
\tikzexternalize
\usepackage{xsim}

\begin{document}

\begin{exercise}
  \begin{tikzpicture}
    \draw (0,0) -- (5,1) ;
  \end{tikzpicture}
\end{exercise}

\end{document}

Although – I wonder if a test-figure0-exercise-1-exercise-body.tex in addition to test-exercise-1-exercise-body is to be expected or if it can be avoided. I'll investigate and will come back to you.

[zoqaeski]

Apologies for the gigantic MWE, though that definitely does produce the issue. It was mainly to illustrate the sheer number of additional files that gets produced on a typical document.

Modifying your MWE so it uses TikZ list and make causes the extra files to be output (the shellesc package is necessary for this to compile with luatex instead of plain latex). I find the list and make option to be much easier on my system, and it has the added advantage that you can generate the images independently of the main document(s).

\RequirePackage{shellesc}
\documentclass{article}

\usepackage{xsim}
\usepackage{tikz}
\usetikzlibrary{external}
\tikzexternalize[shell escape=-enable-write18]
\tikzset{external/mode=list and make}

\begin{document}

\begin{exercise}
  \begin{tikzpicture}
    \draw (0,0) -- (5,1);
  \end{tikzpicture}
\end{exercise}

\end{document}

For this example the build procedure is latex, make -f \jobname.makefile, latex; using lualatex instead of latex produces the same results.

[pablgonz]

This may not be a solution as such, but, with the new [no-files] option you can generate your example file, assuming that the tikz folder exists this file compiles without problems and without generating so many unused files.

% arara: lualatex: { synctex: true, shell: true }
% arara: tikzmake: { options: ['-j2'] }
% arara: lualatex: { synctex: true, shell: true }
\documentclass{article}
\usepackage{shellesc}
%% e-TeX tools
\usepackage{etoolbox}
%% Maths typography — re-enable these if we need maths support
\usepackage{xfrac}
\usepackage{physics}
\usepackage{mathtools}
\usepackage[no-files]{xsim} %
\xsimsetup{
  exercise/within = part,
  exercise/print = true,
  solution/print = false,
  print-solutions/headings = true,
}
\usepackage{siunitx}
\sisetup{%
  detect-all,
  per-mode=symbol,
  bracket-unit-denominator=false
}
\DeclareSIPostPower\inv{-1}
% \usepackage{unicode-math} % not work after update TeXLive
\usepackage{tikz}
\usetikzlibrary{calc}
\usetikzlibrary{decorations.markings}
\usetikzlibrary{decorations.pathmorphing}
\usetikzlibrary{decorations.pathreplacing}
\usetikzlibrary{decorations.shapes}
\usetikzlibrary{intersections}
\usetikzlibrary{positioning}
\usetikzlibrary{patterns}
\usepgflibrary{arrows.meta}
\usetikzlibrary{bending}
\usetikzlibrary{angles}
\usetikzlibrary{hobby}
\usetikzlibrary{external}
\tikzexternalize[prefix=tikz/,shell escape=-enable-write18]
\tikzset{external/system call={lualatex \tikzexternalcheckshellescape -halt-on-error -interaction=scrollmode -jobname "\image" "\texsource"}}
\tikzset{external/mode=list and make}

%% Define all TikZ styles here
\tikzset{%
  dim edge/.style={%
    densely dashed, line width=0.5pt, shorten >= -2.5mm, color=black!75!white
  },
  dim length/.style={
    draw, line width=0.5pt, arrows={Stealth-Stealth}, 
    every node/.style={above=2pt, midway, fill=white, node font=\footnotesize, inner sep=1pt},
  },
  force/.style={draw, line width=1pt, arrows = {Stealth-}},   % Force with arrow tip at start
  force'/.style={draw, line width=1pt, arrows = {-Stealth}},  % Force with arrow tip at end
  pin support/.pic={
    \draw[line width=1pt] (0, 0) -- ++(0.2,-0.5) -- ++(-0.4, 0) -- cycle;
    \draw[line width=1pt, fill=white] (0, 0) circle[radius=0.10];
    \fill[black!20] (-0.4, -0.5) -- ++(0.8, 0) -- ++(0, -0.4) -- ++(-0.8, 0) -- cycle;
    \draw[line width=1pt] (-0.4, -0.5) -- ++(0.8, 0);
  },
  roller support/.pic={
    \draw[line width=1pt] (0, 0) -- ++(0.2,-0.30) -- ++(-0.4, 0) -- cycle;
    % \draw[line width=1pt, fill=white] (0, 0) circle[radius=0.10];
    \draw[line width=1pt, fill=white] (-0.10, -0.4) circle[radius=0.10];
    \draw[line width=1pt, fill=white] (0.10, -0.4) circle[radius=0.10];
    \fill[black!20] (-0.4, -0.5) -- ++(0.8, 0) -- ++(0, -0.4) -- ++(-0.8, 0) -- cycle;
    \draw[line width=1pt] (-0.4, -0.5) -- ++(0.8, 0);
  },
  fixed support/.pic={
    \fill[black!20] (-0.4, 0) -- ++(0.8, 0) -- ++(0, -0.4) -- ++(-0.8, 0) -- cycle;
    \draw[line width=1pt] (-0.4, 0) -- ++(0.8, 0);
  },
  pin connection/.pic={
    \draw[line width=1pt, fill=white,pic actions] (0, 0) circle[radius=0.10];
  },
  fixed connection/.pic={
    \path[pic actions] (0, 0) -- (0.25, 0) -- (0, -0.25) -- cycle;
  },
  % png export,
}

\begin{document}
\begin{exercise}
  Draw the deflection of the following beam:
  \begin{center}
    \tikzsetnextfilename{week3-1}
    \begin{tikzpicture}
      \coordinate (A) at (0, 0);
      \coordinate (D) at ($(A) + (8, 0)$);
      \coordinate (B) at ($(A)!0.3!(D)$);
      \coordinate (C) at ($(A)!0.7!(D)$);
      \path pic at (B) {roller support};
      \path pic at (C) {roller support};
      \draw[line width=3pt, blue!75] (A) -- (D);
      \path pic at (A) {pin support};
      \path pic at (D) {pin support} ;
      \node[above=1mm] at (A) {A};
      \node[above=1mm] at (B) {B};
      \node[above=1mm] at (C) {C};
      \node[above=1mm] at (D) {D};
      \draw[force] (4, 0) -- ++(0, 1) node[above] {\textbf{P}};
    \end{tikzpicture}
  \end{center}
\end{exercise}

\begin{solution}
  \begin{center}
    \tikzsetnextfilename{week3-1a}
    \begin{tikzpicture}
      \coordinate (A) at (0, 0);
      \coordinate (D) at ($(A) + (8, 0)$);
      \coordinate (B) at ($(A)!0.3!(D)$);
      \coordinate (C) at ($(A)!0.7!(D)$);
      \path pic at (B) {roller support};
      \path pic at (C) {roller support};
      \draw[line width=3pt, blue!75] (A) -- (D);
      \path pic at (A) {pin support};
      \path pic at (D) {pin support} ;
      \node[above=1mm] at (A) {A};
      \node[above=1mm] at (B) {B};
      \node[above=1mm] at (C) {C};
      \node[above=1mm] at (D) {D};
      \draw[force] ($(A)!0.5!(D)$) -- ++(0, 1) node[above] {\textbf{P}};

      \draw[force', red] (A) -- ++(0, -1) node[below] {$V_A$};
      \draw[force, red] (B) -- ++(0, -1) node[below] {$V_B$};
      \draw[force, red] (C) -- ++(0, -1) node[below] {$V_C$};
      \draw[force', red] (D) -- ++(0, -1) node[below] {$V_D$};
      \draw[red, line width=1pt] (A) to[curve through={(B) .. ($(B)!0.5!(C) - (0, 0.5)$) .. (C)}] (D);
    \end{tikzpicture}
  \end{center}
\end{solution}

\begin{exercise}
  Draw the deflection of the following beam:
  \begin{center}
    \tikzsetnextfilename{week3-2}
    \begin{tikzpicture}
      \coordinate (A) at (0, 0);
      \coordinate (D) at ($(A) + (8, 0)$);
      \coordinate (B) at ($(A)!0.3!(D)$);
      \coordinate (C) at ($(A)!0.7!(D)$);
      \path pic at (B) {roller support};
      \path pic at (C) {roller support};
      \draw[line width=3pt, blue!75] (A) -- (D);
      \path pic at (A) {pin support};
      \path pic at (D) {pin support} ;
      \node[above=1mm] at (A) {A};
      \node[above=1mm] at (B) {B};
      \node[above=1mm] at (C) {C};
      \node[above=1mm] at (D) {D};
      \draw[force] ($(A)!0.5!(B)$) -- ++(0, 1) node[above] {\textbf{P}};
    \end{tikzpicture}
  \end{center}
\end{exercise}

\begin{solution}
  \begin{center}
    \tikzsetnextfilename{week3-1b}
    \begin{tikzpicture}
      \coordinate (A) at (0, 0);
      \coordinate (D) at ($(A) + (8, 0)$);
      \coordinate (B) at ($(A)!0.3!(D)$);
      \coordinate (C) at ($(A)!0.7!(D)$);
      \path pic at (B) {roller support};
      \path pic at (C) {roller support};
      \draw[line width=3pt, blue!75] (A) -- (D);
      \path pic at (A) {pin support};
      \path pic at (D) {pin support} ;
      \node[above=1mm] at (A) {A};
      \node[above=1mm] at (B) {B};
      \node[above=1mm] at (C) {C};
      \node[above=1mm] at (D) {D};
      \draw[force] ($(A)!0.5!(B)$) -- ++(0, 1) node[above] {\textbf{P}};

      \draw[force, red] (A) -- ++(0, -1) node[below] {$V_A$};
      \draw[force, red] (B) -- ++(0, -1) node[below] {$V_B$};
      \draw[force', red] (C) -- ++(0, -1) node[below] {$V_C$};
      \draw[force', red] (D) -- ++(0, -1) node[below] {$V_D$};
      \draw[red, line width=1pt] (A) to[curve through={($(A)!0.5!(B) - (0, 0.5)$) .. (B) .. (C)}] (D);
      % \draw[purple, line width=1pt] (A) to[curve through={($(A)!0.5!(B) - (0, 0.5)$)}] (B);
      % \draw[red, line width=0.5pt] ($(A)!0.5!(B)$) -- ++(0, -1.5);
      % \draw ($(A)!0.5!(B) - (0, 0.5)$) circle[radius=0.1];
    \end{tikzpicture}
  \end{center}
\end{solution}

\begin{exercise}
  Draw the deflection of the following beam:
  \begin{center}
    \tikzsetnextfilename{week3-3}
    \begin{tikzpicture}
      \coordinate (A) at (0, 0);
      \coordinate (D) at ($(A) + (8, 0)$);
      \coordinate (B) at ($(A)!0.3!(D)$);
      \coordinate (C) at ($(A)!0.7!(D)$);
      \path pic at (B) {roller support};
      \path pic at (C) {roller support};
      \draw[line width=3pt, blue!75] (A) -- (D);
      \path pic at (A) {pin support};
      \path pic at (D) {pin support} ;
      \node[above=1mm] at (A) {A};
      \node[above=1mm] at (B) {B};
      \node[above=1mm] at (C) {C};
      \node[above=1mm] at (D) {D};
      \draw[force] ($(A)!0.5!(B)$) -- ++(0, 1) node[above] {$\textbf{P}_1$};
      \draw[force] ($(C)!0.5!(D)$) -- ++(0, 1) node[above] {$\textbf{P}_2$};
    \end{tikzpicture}
  \end{center}
\end{exercise}

\begin{solution}
  \begin{center}
    \tikzsetnextfilename{week3-1c}
    \begin{tikzpicture}
      \coordinate (A) at (0, 0);
      \coordinate (D) at ($(A) + (8, 0)$);
      \coordinate (B) at ($(A)!0.3!(D)$);
      \coordinate (C) at ($(A)!0.7!(D)$);
      \path pic at (B) {roller support};
      \path pic at (C) {roller support};
      \draw[line width=3pt, blue!75] (A) -- (D);
      \path pic at (A) {pin support};
      \path pic at (D) {pin support} ;
      \node[above=1mm] at (A) {A};
      \node[above=1mm] at (B) {B};
      \node[above=1mm] at (C) {C};
      \node[above=1mm] at (D) {D};
      \draw[force] ($(A)!0.5!(B)$) -- ++(0, 1) node[above] {$\textbf{P}_1$};
      \draw[force] ($(C)!0.5!(D)$) -- ++(0, 1) node[above] {$\textbf{P}_2$};

      \draw[force, red] (A) -- ++(0, -1) node[below] {$V_A$};
      \draw[force, red] (B) -- ++(0, -1) node[below] {$V_B$};
      \draw[force, red] (C) -- ++(0, -1) node[below] {$V_C$};
      \draw[force, red] (D) -- ++(0, -1) node[below] {$V_D$};
      \draw[red, line width=1pt] (A) to[curve through={($(A)!0.5!(B) - (0, 0.5)$) .. (B) .. (C) .. ($(C)!0.5!(D) - (0, 0.5)$)}] (D);
    \end{tikzpicture}
  \end{center}
\end{solution}

\begin{exercise}
  Using the \emph{force method} to determine the deflection at $C$ of the
  following portal frame. The cross sections of the truss members is given by
  $C_1$, that of the horizontal beam members by $C_2$, and that of the vertical
  members by $C_3$.
  \begin{center}
    \tikzsetnextfilename{week5-portal-frame}
    % \tikzexternaldisable
    \begin{tikzpicture}
      \draw[line width=3pt, blue!75] (0, 0) coordinate (node-A) node[left, text=black] {$A$}
      -- ++(0, 6) coordinate (node-B) node[above left, text=black] {$B$}
      -- ++(2.75, 0) coordinate (node-D) node[above right, text=black] {$D$}
      -- ++(2.75, 0) coordinate (node-F) node[above right, text=black] {$F$}
      -- ++(0, -6) coordinate (node-G) node[left, text=black] {$G$};

      \draw[line width=2pt, blue!75] (node-B)
      -- ($(node-B)!0.5!(node-D) - (0, 2)$) coordinate (node-C) node[below, text=black] {$C$}
      -- (node-D) 
      -- ($(node-D)!0.5!(node-F) - (0, 2)$) coordinate (node-E) node[below, text=black] {$E$}
      -- (node-F) 
      (node-C) -- (node-E); 

      \draw pic[draw, line width=1.5pt, blue!75] at ($(node-B)!0.07!(node-C)$) {pin connection};
      \draw pic[draw, line width=1.5pt, blue!75] at (node-C) {pin connection};
      \draw pic[draw, line width=1.5pt, blue!75] at ($(node-C)!0.5!(node-E)!0.95!(node-D)$) {pin connection};
      \draw pic[draw, line width=1.5pt, blue!75] at (node-E) {pin connection};
      \draw pic[draw, line width=1.5pt, blue!75] at ($(node-F)!0.07!(node-E)$) {pin connection};

      \draw[force] (node-D) -- ++(0, 1) node[above] {\SI{5}{\kN}};

      \path pic at (node-A) {pin support};
      \path pic at (node-G) {roller support};

      \draw[dim edge] (node-A) -- ++(0, -1) coordinate (node-A-dim-down);
      \draw[dim edge] (node-B) -- ++(0, 0.5) coordinate (node-B-dim-up);
      \coordinate (node-D-dim-up) at ($(node-D) + (0, 0.5)$);
      \draw[dim edge] (node-F) -- ++(0.5, 0) coordinate (node-F-dim-right);
      \draw[dim edge] (node-G) -- ++(0.5, 0) coordinate (node-G-dim-right);
      \draw[dim edge] (node-G) -- ++(0, -1) coordinate (node-G-dim-down);
      \draw[dim edge] (node-A) -- (node-C |- node-A) coordinate (node-A-dim-inner);
      \draw[dim edge] (node-C) -- (node-A |- node-C) coordinate (node-C-dim-inner);

      \draw[dim length] (node-A-dim-down) -- (node-G-dim-down) node[below] {\SI{550}{\mm}};
      \draw[dim length] (node-B-dim-up) -- (node-D-dim-up) node[above] {\SI{275}{\mm}};
      \draw[dim length] (node-F-dim-right) -- (node-G-dim-right) node[right] {\SI{600}{\mm}};
      \draw[dim length] ($(node-A)!0.5!(node-A-dim-inner)$) -- ($(node-C-dim-inner)!0.5!(node-C)$) node[right] {\SI{400}{\mm}};

      \coordinate (c1-top-left) at ($(node-F) + (3, 0)$);

      \draw[line width=1pt, blue, fill=blue!50] (c1-top-left) 
      -- ++(0.45, 0) coordinate (c1-top-right)
      -- ++(0, -0.45) coordinate (c1-bottom-right)
      -- ++(-0.45, 0) coordinate (c1-bottom-left)
      -- cycle;

      \node at ($(c1-top-left)!0.5!(c1-bottom-left) - (0.5, 0)$) {$C_1$};

      \draw[dim edge] (c1-top-left) -- ++(0, 0.25) coordinate (c1-left-edge);
      \draw[dim edge] (c1-top-right) -- ++(0, 0.25) coordinate (c1-right-edge);
      \draw[dim edge] (c1-top-right) -- ++(0.25, 0) coordinate (c1-top-edge);
      \draw[dim edge] (c1-bottom-right) -- ++(0.25, 0) coordinate (c1-bottom-edge);

      \draw[dim length] (c1-left-edge) -- (c1-right-edge) node[above=1mm] {\SI{15}{\mm}};
      \draw[dim length] (c1-top-edge) -- (c1-bottom-edge) node[right=1mm] {\SI{15}{\mm}};

      \coordinate (c2-bottom-left) at ($(node-F)!0.5!(node-G) + (3, 0)$);

      \draw[line width=1pt, blue, fill=blue!50] (c2-bottom-left)
      -- ++(0, 0.9) coordinate (c2-top-left)
      -- ++(0.9, 0) coordinate (c2-top-right)
      -- ++(0, -0.9) coordinate (c2-bottom-right)
      -- cycle;

      \node at ($(c2-top-left)!0.5!(c2-bottom-left) - (0.5, 0)$) {$C_2$};

      \draw[dim edge] (c2-top-left) -- ++(0, 0.25) coordinate (c2-left-edge);
      \draw[dim edge] (c2-top-right) -- ++(0, 0.25) coordinate (c2-right-edge);
      \draw[dim edge] (c2-top-right) -- ++(0.25, 0) coordinate (c2-top-edge);
      \draw[dim edge] (c2-bottom-right) -- ++(0.25, 0) coordinate (c2-bottom-edge);

      \draw[dim length] (c2-left-edge) -- (c2-right-edge) node[above=1mm] {\SI{30}{\mm}};
      \draw[dim length] (c2-top-edge) -- (c2-bottom-edge) node[right=1mm] {\SI{30}{\mm}};

      \coordinate (c3-bottom-left) at ($(node-G) + (3, 0)$);

      \draw[line width=1pt, blue, fill=blue!50] (c3-bottom-left)
      -- ++(0, 1.2) coordinate (c3-top-left)
      -- ++(1.2, 0) coordinate (c3-top-right)
      -- ++(0, -1.2) coordinate (c3-bottom-right)
      -- cycle;

      \node at ($(c3-top-left)!0.5!(c3-bottom-left) - (0.5, 0)$) {$C_3$};

      \draw[dim edge] (c3-top-left) -- ++(0, 0.25) coordinate (c3-left-edge);
      \draw[dim edge] (c3-top-right) -- ++(0, 0.25) coordinate (c3-right-edge);
      \draw[dim edge] (c3-top-right) -- ++(0.25, 0) coordinate (c3-top-edge);
      \draw[dim edge] (c3-bottom-right) -- ++(0.25, 0) coordinate (c3-bottom-edge);

      \draw[dim length] (c3-left-edge) -- (c3-right-edge) node[above=1mm] {\SI{40}{\mm}};
      \draw[dim length] (c3-top-edge) -- (c3-bottom-edge) node[right=1mm] {\SI{40}{\mm}};

    \end{tikzpicture}
  \end{center}
\end{exercise}

\begin{solution}
  This is a trick question, there is no solution to type out right now.
\end{solution}

\printsolutionstype{exercise}

\end{document}

[cgnieder]

Referring to https://github.com/cgnieder/xsim/issues/32#issuecomment-396874149 I still see no difference. I compile it once with

\usepackage{xsim}

and once with

\usepackage[no-files]{xsim}

and the difference is only the files produced this by xsim itself. I close this as it seems this is not a bug or issue in xsim itself.