Why is grattex so slow??

HughParsonage commented 6 years ago

General issue for performance concerns:

Issue

Stan manual (660 pages) takes 4 seconds to compile, similar timings for other books. AP report takes > 40 seconds in batchmode and >100 seconds (15+14+35+35) to run fully. ShareLaTeX timings are likely to be higher still.

Possible bottlenecks

biber/biblatex is a known bottleneck vis-à-vis bibtex. Main issue with conversion would be stricter tests for encoding and nastier programming at the cls file level to coerce to Grattan style.
PowerPoint PDF production is not linearized nor standardized and uses a different PDF specification. PDFs generated by ghostscript are slightly faster but not obviously so.
Footnotes are effectively 'pages' and conceal the length of documents compared to books etc.
Memory consumption is high.

HughParsonage commented 6 years ago

Authors can avoid the biber/biblatex runs and re-runs by just commenting out \addbibresource and \printbibliography. Anecdotally this results in ten-fold reduction in compile time.

HughParsonage commented 6 years ago

Some examples using the Housing Affordability report:

One run of pdflatex: 47s No bibliography, no \addbibresource: 38s \textcite, \footcite disabled: 22s graphicx to draft: 20s

So it seems that the bottleneck really is the biblography (note that these examples are actually favourable towards bibliography speed, because biber is not included).

The challenge is a safe and reasonable way to disable \textcite etc.

HughParsonage commented 6 years ago

In addition, on the question of whether it is feasible to move to bibtex, the highest code-points by line:

        char  int     N
 1:        }  125 19812
 2:        {  123  6748
 3:        ’ 8217    51
 4:        ~  126    37
 5:        – 8211    22
 6:        é  233    19
 7:        ö  246     4
 8: <U+200E> 8206     4
 9:        á  225     3
10:        ä  228     3
11:        - 8208     2
12:        L  321     2
13:           160     2
14:        — 8212     2
15:        ó  243     2
16:        … 8230     1
17:        ” 8221     1
18:        Å  197     1
19:        è  232     1
20:        ø  248     1
21:        ü  252     1
        char  int     N

i.e. about 85 / 26,719 lines have non-ASCII codes. In the new, compact bibliography there are none.

HughParsonage commented 6 years ago

Both slower and difficult to gain speed on ShareLaTeX vs locally: 95s and 90s vs 15s and 10s locally...

HughParsonage commented 6 years ago

No real benefit to backend=bibtex: 16 s vs 17 s.

Turning off \CenturyFootnote logging and reruns has minimal effect once the document is processed the first time.

grattan / grattex

Why is grattex so slow?? #111

Issue

Possible bottlenecks