[dev feature] Use actions to check build system on UNIX and Windows

[funnym0nk3y]

As I filed the issue #1897 it became clear that there are difficulties to make the build system work on different platforms with different configurations. As I updated my version of this extension to 8.6.0 I am having some strange issues with the build system (no filed bug yet, I have to first figure out what is happening). To improve reliability and add some testing I am suggesting to use github actions to run some tests.

Looking forward to your comments!

funnym0nk3y

[jlelong]

Unfortunately, easier said than done. The real difficulty is to design the tests considering the wide variety of configurations. It goes far beyond the Windows or Unix dichotomy. I would love to see automated non-regression tests.

[tamuratak]

Because we can override user settings with .vscode/settigns.json in workspaces, we can test "the wide variety of configurations" by creating directories having .vscode/settigns.json corresponding to each test case.

[jlelong]

Of course we can use local configurations. I was more referring to the combinatorial aspects (OS, Docker, remote, LaTeX distribution, special characters, ...)

[tamuratak]

On GitHub Actions, we can choose Ubuntu, Windows, and macOS as OS with free. On each OS, lots of software are already installed.

• https://help.github.com/en/actions/automating-your-workflow-with-github-actions/software-installed-on-github-hosted-runners

I have tried installing TeX Live on Windows on GitHub Actions. It seems to work.

• https://github.com/tamuratak/github-actions-examples/tree/texlive

On Windows, we can use Chocolatey, a package manager for Windows, already installed. The MikTeX package is available.

[funnym0nk3y]

@tamuratak You are way more proficient with github actions than I am. I will try to follow your steps, but I can't guarantee I can contribute...

@jlelong So what should be tested?

• OS: Windows, Linux, macOS
• Multiple build systems: pdflatex, lualatex through latexmk
• Multiple distributions (if available): TeXLive, MikTeX

What do you mean by docker and special characters?

[jlelong]

• build systems:
  • latexmk with one of the following options at a time: -pdf, -lualatex, -xelatex. This can be set in a .latexmkrc file
  • it is also important to consider different -output-directory (variable $out_dir in .latexmkrc)
• Among all, we should also consider documents using the subfiles package. See #1895 for examples of such documents.

  What do you mean by docker and special characters?

Most of the problems we had recently were related to using a dockerized LaTeX distribution.

[tamuratak]

I have added an example of running latexmk on GitHub Actions.

• https://github.com/tamuratak/github-actions-examples/tree/texlive

You can see the familiar log.

[tamuratak]

@jlelong Have you changed your mind on adding tests from the time? As I said before, if adding tests harms your motivation, I do not push that strongly. I will close this issue.

[jlelong]

I am afraid I was not clear at the time. I would be delighted to have automated tests, they would be so helpful to check all the different situations with subfiles for instance. My point is that unfortunately I do not have the time to design all this. @tamuratak if you can find the time to implement the whole framework, I would be happy to contribute.

[tamuratak]

Great. I think aiming at adding comprehensive unit tests is not feasible. I will start by adding integration tests. On GitHub Actions,

1. VS Code automatically opens test directories as workspace and calls a build command.
2. We check whether pdf files are generated.