PHPirates / travis-ci-latex-pdf

Overview of different methods to build LaTeX with GitHub Actions or Travis-CI (idea by @jackolney but completely rewritten by @PHPirates and contributors).
https://tex.stackexchange.com/a/398831/98850
MIT License
110 stars 23 forks source link
continuous-integration docker github-actions github-releases gitlab-ci latex latexmk lualatex miniconda pdflatex tectonic travis-ci xelatex

LaTeX + Git + Travis → release pdf

Build Status Gitter

Write LaTeX, push to git, let Travis automatically build your file and release a pdf automatically to GitHub releases when the commit was tagged.

This repository contains an overview of the methods you could use to build your LaTeX on a remote server (continuous integration server). You want that because then every time you push it will automatically check if you pushed valid LaTeX.

If you are looking for instructions to build LaTeX locally, look here. If you are looking for instructions to build LaTeX on GitLab CI, have a look at the GitLab section. If you are looking for instructions to build ConTeXt, have a look at the ConTeXt section.

Quickstart

This repository contains a huge amount of information, and as you can see I have worked with a lot of different options. These are my two favourites:

Table of Contents

GitHub Actions

In 2019 GitHub introduced GitHub Actions, which is essentially an alternative to Travis.

To use it:

For more information, see https://help.github.com/en/actions/automating-your-workflow-with-github-actions

In the following sections we will give an overview and comparison of available GitHub Actions to build LaTeX. Note however, that you can adapt any of the methods for using Travis CI to work with GitHub Actions, because the syntax for the yml configuration files is very similar.

Docker image with full TeX Live

Currently we know the following Actions.

Github Action for LaTeX by xu-cheng

Advantages:

Disadvantages:

Example in this repo: xu-cheng-docker-full-texlive.yml

Build time example files: 3-4 min.

LaTeX compilation by dante-ev

This is a fork of Github Action for LaTeX by xu-cheng, of which the only changes are now also in the original Action by xu-cheng.

Latex-multicompiler by Jatus93

Fork of 'LaTeX compilation by dante-ev' but reads the file to compile from a .fileToCompile file in your repository so this does not seem to have any advantages.

Github Actions for build LaTeX and release pdf by MaineK00n

Uses https://github.com/Paperist/docker-alpine-texlive-ja which installs the TeX Live package collection-langjapanese by default and otherwise does not seem to have any advantages (or documentation at all).

Uberblatt by ottojo

debian-based Docker image with a full texlive installed using the debian package. Includes pygments and pandoc, for source see https://hub.docker.com/r/aergus/latex/dockerfile. No known advantages currently.

Docker image with Tectonic

The following Actions use Tectonic, which means they have the advantages and disadvantages of Tectonic as described below

Using the Docker image directly

Essentially this just executes docker run --mount src=$GITHUB_WORKSPACE/src,target=/usr/src/tex,type=bind dxjoke/tectonic-docker /bin/sh -c "tectonic main.tex" to compile main.tex in src.

Build time example file: 30s

setup-tectonic by WtfJoke

This Github Action downloads tectonic (and optionally biber) from the latest official releases. It has a significant advantage over using the Docker image directly: it supports caching of installed LaTeX packages so it is significantly faster for documents with many dependencies. For the documentation and more explanation (including one about the cache) see https://github.com/wtfjoke/setup-tectonic

Example in this repo: .github/workflows/setup-tectonic-action.yml

Build time example file: 20s

Compile LaTeX by vinay0410

Uses a fork from https://github.com/WtfJoke/tectonic-docker with no significant changes.

Advantages:

Disadvantages:

Example in this repo: vinay04010-docker-tectonic.yml

Build time example files: 4-5 min.

TeX Live

Installing TeX Live directly

You can do basically the same as 4-texlive/.travis.yml for GitHub Actions. The workflow .github/workflows/texlive.yml contains a full example of installing a basic TeX Live installation and using texliveonfly and caching to install the rest. It also contains the steps needed to automatically release a pdf to GitHub if you tag a commit, and also shows that it is easy to extend with other programs like pythontex. The complete time the GitHub Actions needs for an empty LaTeX file is only 20 seconds.

The steps to use this are then:

paper-maker by andycasey using Ubuntu texlive packages

This actions installs a default set of texlive packages from Ubuntu, at the moment texlive-publishers texlive-latex-recommended texlive-latex-extra texlive-fonts-recommended texlive-fonts-extra.

Then it compiles using pdflatex and bibtex, using -shell-escape.

By default, you don't specify any options, but for customizations you can copy the source yaml from https://github.com/andycasey/paper-maker/blob/master/action.yml and modify it.

Advantages:

Disadvantages:

Because it would be so much work figuring out what packages are needed, especially when TeX Live was installed using the Ubuntu packages, no example file is provided with this repository.

LaTeX linting

LaTeX linter (chktex) by j2kun

Uploading the pdf to a GitHub Release when a tag is pushed

Basically see https://github.com/actions/upload-release-asset#example-workflow---upload-a-release-asset, example in this repo in release-pdf-on-tag.yml.

Overview and comparison of methods to build LaTeX using Travis CI

Tectonic

Thanks to Dan Foreman-Mackey for writing about Tectonic. The first two methods do not use the pdflatex engine to compile, but Tectonic which is a fork of XeTeX (thanks to ShreevatsaR for pointing this out). Please leave a comment to vote for adding documentation about using Tectonic at https://github.com/tectonic-typesetting/tectonic/issues/373

Important note: there have been recurring problems with the persistent urls from which the packages are downloaded, if you encounter this (e.g. 'PURL setup link has expired SSL certificate' or 'couldn't probe ...') then you can try the workaround described at https://github.com/tectonic-typesetting/tectonic/issues/131#issuecomment-435792227

Pro:

Con:

We will quickly compare two methods to use Tectonic, after that we will discuss more conventional methods which can compile with pfdlatex, lualatex etc.

Note that since May 2018 Travis moved open source to travis-ci.com (instead of travis-ci.org). Travis is also now available as GitHub App in the Marketplace, but old repositories (like this one) are still on travis-ci.org.

1. Docker image with Tectonic

Thanks to Norbert Pozar (@rekka) for providing the original Docker image. Manuel (@WtfJoke) extended it by integrating biber. Docker provides the ability to download a pre-installed Tectonic and then run it on your LaTeX files.

Pro:

Con:

Build time example file: two minutes

Want this? Instructions below.

2. Miniconda with Tectonic

Pro:

Con:

Build time example file: 1-2 minutes

Want this? Instructions below.

3. Docker image with TeX Live

This method downloads a docker image which contains a small TeX Live installation. Thanks to Andreas Strauman for figuring this out, give his tex.stackexchange.com answer an upvote if you like it! At the moment it uses latexmk to compile pdfs which is configured to run pdflatex by default, but it should be easy to configure it for other tex engines.

Pro:

Con:

Build time example file: 4 minutes (9 minutes when using texliveonfly)

Want this? Instructions below.

4. TeX Live with pdflatex/lualatex/latexmk/xelatex/texliveonfly/etc

Thanks to Joseph Wright who pointed out that they use something based on this setup for LaTeX3 development.

Pro:

Con:

Build time example file: 1-2 minutes

Want this? Instructions below.

5. TeX Live and pdflatex via tinytex with R.

Thanks to Hugh for pointing out this option.

Pro:

Con:

Build time example file: 5-8 minutes

Want this? Instructions below.

Instructions for each build method

Instructions for building with Docker and Tectonic

Instructions for building with Miniconda and Tectonic

Separate instructions for adding biber to your Miniconda and Tectonic setup

These changes have already been added to the .travis.yml, but to be clear here are the separate instructions if you already have Miniconda and Tectonic running:

tectonic --keep-intermediates --reruns 0 ./main.tex
biber main
tectonic ./main.tex

Instructions for building with TeX Live from a Docker image

Using a docker image with texlive and texliveonfly

You can also use a docker image which uses texliveonfly, it is based on the above-mentioned docker image. It is somewhat experimental in the sense that sometimes texliveonfly seems to hang (for longer than 10 mins), although usually it seems to work fine. Texliveonfly calls pdflatex by default, but you can change this and more options in arguments. Some documentation can be found at https://tex.stackexchange.com/a/463842/98850 which also includes a little list of packages which texliveonfly does not download.

The only change in the instructions is that you use the docker image as specified in texlive-docker-texliveonfly/.travis.yml, and that you don't need to specify packages manually in that file.

The docker source file of this image can be found in https://github.com/Strauman/travis-latexbuild/tree/master/docker/texliveonfly

Github Package Registry

Instead of downloading from Docker Hub, you can also download the image from the Github Package Registry. You can see it at https://github.com/Strauman/travis-latexbuild/packages/92067?version=texliveonfly-2019

This requires you to upload your username and a github token as Travis environment variables.

Just replace the script: block with

script:
  - docker login -u $GH_REGISTRY_USERNAME -p $GH_REGISTRY_TOKEN docker.pkg.github.com
  - docker run --mount src=$TRAVIS_BUILD_DIR/,target=/repo,type=bind docker.pkg.github.com/strauman/travis-latexbuild/texlive-latexbuild:texliveonfly-2019

Instructions for building with pdflatex/lualatex/latexmk/xelatex/texliveonfly/etc and TeX Live

If for some reason you prefer the pdflatex (or any other) engine with the TeX Live distribution, read on. This is based on the LaTeX3 build file.

This method installs an almost minimal TeX Live installation on Travis, and compiles with pdflatex. This repo contains:

Instructions

Note that sometimes tlmgr selects a broken mirror to download TeX Live from, so you get an error like Output was gpg: verify signatures failed: eof. Restarting the build will probably fix this, it will auto-select a different mirror. (Thanks to @jason-neal for testing this.)

You can also read a nice blog post by Jeremy Grifski (@jrg94) about using the setup from this repo including the minted package at https://therenegadecoder.com/code/how-to-build-latex-with-travis-ci-and-minted/

Using custom fonts

In order to enable Travis to use custom fonts, i.e. those fonts not provided by Ubuntu or TeX Live packages, it is probably easiest to push them with git. An example is in this repo, with the file src/fonts.tex and the fonts in src/fonts. You can do the following to use them on Travis:

Instructions for building with TeX Live and pdflatex via tinytex with R

To automatically deploy pdfs to GitHub release

First time setup

We will add a configuration to the .travis.yml such that a pdf will be automatically uploaded to GitHub releases when you tag a commit, also see the Travis documentation.

Tips

Markdown:

[![Build Status](https://api.travis-ci.com/username/reponame.svg)](https://travis-ci.com/username/reponame)

reStructuredText:

.. image:: https://travis-ci.com/username/reponame.svg?branch=master
    :target: https://travis-ci.com/username/reponame
    :alt: Build Status

Now you can for example get the hash of the commit with \gitHash. An example can be found in src/git-info-2-mwe.tex. For more info, see the documentation at http://mirrors.ctan.org/macros/latex/contrib/gitinfo2/gitinfo2.pdf (Thanks to @shabbychef for mentioning this)

Troubleshooting

Notes

I also put some of these instructions on the TeX Stackexchange.

Some original thoughts from harshjv's blog, and thanks to jackolney for all his attempts to put it into practice. Also see harshjv's original blog post.

Contributing

If you want to add/update a method to build LaTeX, look in the contributing guidelines.

GitLab CI

At the moment, only the Texlive method explained above was tested in GitLab for this repo. The instructions are similar but with of course a different configuration file, which can be found in gitlab/texlive for now. This still has to be moved to GitLab. The only disadvantage is that the caching of TeX Live does not work, because it is not installed in the project, but artifacts have to be in the project directory for GitLab. The build time for the example file is around four minutes.

A more extensive overview of configurations for GitLab CI will come in the future, either in this or an other repo. Please contribute if you can help with this! Some links to get started:

https://tex.stackexchange.com/questions/459484/compiling-latex-files-automatically-with-gitlab-ci

https://www.vipinajayakumar.com/continuous-integration-of-latex-projects-with-gitlab-pages.html

https://sayantangkhan.github.io/latex-gitlab-ci.html

https://tex.stackexchange.com/questions/412740/gitlab-ci-runner-with-relative-paths-in-main-tex

https://tex.stackexchange.com/questions/437553/gitlab-ci-using-miktex-docker-image

ConTeXt

Following a TeX Stackexchange answer from the TeXnician and the official docs at https://wiki.contextgarden.net/ConTeXt_Standalone#Single_user_installation it is not too difficult to install ConTeXt (and cache the installation).

As an alternative, you could use a Docker image, for example one from https://gitlab.com/islandoftex/images/context

In this repository there is an example for Github Actions, see .github/workflows/context.yml, but it should be easy to extend to other build systems as well.

Usage of the Github Action

This actions caches the ConTeXt installation. Build time example file: around 30-40 sec.