Version-stable CTAN repo for the TinyTeX install

riccardoporreca commented 4 years ago

Fixes #169.
TeXLive release bound to the base Debian image.
CTAN repo exposed as build argument and set persistently via ENV. The default is on ENV rather than ARG to allow going back to the nominal latest repo with empty argument: docker build --build-arg CTAN_REPO= -t rocker/verse:latest-repo .
Backported down to verse:3.4.3.
Use http for TeXLive 2016: install-tl did not support https repositories.
Work around known non-breaking errors of tlmgr path add in TeXLive 2016 (https://tex.stackexchange.com/a/314079).

riccardoporreca commented 4 years ago

All images built locally via

docker build --pull -t rocker/verse:3.6.1-texlive-stable .
for tag in 3.4.3 3.4.4 3.5.0 3.5.1 3.5.2 3.5.3 3.6.0 devel; \
do \
  docker build --pull -t rocker/verse:$tag-texlive-stable $tag; \
done

Summary of the setup for the various images collected via

for tag in 3.4.3 3.4.4 3.5.0 3.5.1 3.5.2 3.5.3 3.6.0 3.6.1 devel; \
do \
  echo rocker/verse:$tag; \
  docker run --rm rocker/verse:$tag-texlive-stable sh -c ' \
      cat /etc/os-release | grep PRETTY_NAME \
      && echo repo: $CTAN_REPO \
      && tex --version | head -n 1 \
  '; \
done

rocker/verse:3.4.3
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:3.4.4
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:3.5.0
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:3.5.1
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:3.5.2
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:3.5.3
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:3.6.0
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:3.6.1
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo: http://www.texlive.info/tlnet-archive/2017/04/13/tlnet
TeX 3.14159265 (TeX Live 2016)
rocker/verse:devel
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
repo: https://www.texlive.info/tlnet-archive/2019/02/27/tlnet
TeX 3.14159265 (TeX Live 2018)

riccardoporreca commented 4 years ago

Tested by producing the PDF report of the SmaRP Shiny app launched from RStudio in e.g. rocker/verse:3:5:3.

cboettig commented 4 years ago

Looks great, 👏 Many thanks for this!

cboettig commented 4 years ago

Just a note: unfortunately, falling back to the 2016 version also introduces a bug in permissions handling that we had reported to TeXLive team and had patched, see https://github.com/rocker-org/rocker-versioned/issues/104 and https://github.com/yihui/tinytex/issues/77#issuecomment-466584510 .

I believe this is what is now biting @annakrystalli in https://github.com/rocker-org/rocker-versioned/issues/172. I'm not sure of a great fix -- we can add rstudio as the owner of /opt/TinyTex but it will still cause things to break for other users... Thoughts?

riccardoporreca commented 4 years ago

@cboettig, I see we can have a problem here... Just to make sure we are on the same page, is it correct that the issue should still be revealed by using the header below, as you mentioned in in https://github.com/rocker-org/rocker-versioned/issues/104#issuecomment-431244852?

---
title: "Untitled"
header-includes:
   - \usepackage{bbm}
output: pdf_document
---

If that is the case, I can only reproduce the problem for rocker/verse:devel, which makes sense since it using TeX Live 2018.

rocker/verse:latest:

docker run --rm -p 8787:8787 -e DISABLE_AUTH=true --user=rstudio rocker/verse:latest R -e 'cat("---\ntitle: Untitled\nheader-includes:\n   - \\usepackage{bbm}\noutput: pdf_document\n---\n", file = "~/test.Rmd") ; rmarkdown::render("~/test.Rmd")'
# [...]
# Output created: test.pdf

rocker/verse:devel:

docker run --rm -p 8787:8787 -e DISABLE_AUTH=true --user=rstudio rocker/verse:devel R -e 'cat("---\ntitle: Untitled\nheader-includes:\n   - \\usepackage{bbm}\noutput: pdf_document\n---\n", file = "~/test.Rmd") ; rmarkdown::render("~/test.Rmd")'
# [...]
# ! LaTeX Error: File `bbm.sty' not found.

Can you please confirm I am not missing anything for the tags before devel, which are based on TeX Live 2016?

cboettig commented 4 years ago

@riccardoporreca no, for some reason bbm.sty is showing up as already installed in rocker/verse:latest, and hence it is not throwing the error, but both the TeX Live 2018 and TeX Live 2016 are impacted by this permissions bug. A slightly more concise test is:

 docker run --rm  --user=rstudio rocker/verse:devel R -e "tinytex::tlmgr_install('bbm')"

you will see two permission-related errors:

chmod(420,/opt/TinyTeX/tlpkg/texlive.tlpdb) failed: Operation not permitted at /opt/TinyTeX/tlpkg/TeXLive/TLUtils.pm line 1205.

and also the error:

add_link_dir_dir: destination /usr/local/bin not writable, no links from /opt/TinyTeX/bin/x86_64-linux.

However, running the above with rocker:verse, you instead see:

tlmgr install: package already present: bbm

so no error occurs. I'm not at all clear why bbm is already installed on :latest, but trying random .sty files off the top of my head or from the archive, I keep finding they are already installed!! so I'm a bit confused myself.

riccardoporreca commented 4 years ago

@cboettig, thanks for details. There are some confusing peculiarities with the behavior of TinyTeX with the TeX Live 2016 snapshot as opposed to the TeXLive 2018 snapshots.

In particular, when we see

tlmgr install: package already present: bbm

it is because tinytex::tlmgr_install() wrongly detects a failure in the result of tlmgr install and hence attempts a tlmgr update before re-trying tlmgr install (https://github.com/yihui/tinytex/blob/ca68595df600c5e4fe83f468b3d17b5f842e7250/R/tlmgr.R#L106-L110)

This is not any special to bbm, see e.g. the same full behavior with fancyhdr, even running as root

docker run --rm rocker/verse:latest R -e "tinytex::tlmgr_install('fancyhdr')"

[...]
[1/1, ??:??/??:??] install: fancyhdr [9k]
tlmgr: package log updated: /opt/TinyTeX/texmf-var/web2c/tlmgr.log
running mktexlsr ...
done running mktexlsr.
tlmgr update --self
[...]
tlmgr: no self-updates for tlmgr available.
tlmgr install fancyhdr
[...]
tlmgr install: package already present: fancyhdr
tlmgr path add
tlmgr: action path returned an error; continuing.
tlmgr: An error has occurred. See above messages. Exiting.

I guess TeXLive 2016 might wrongly report a failure of tlmgr install, similar to tlmgr path add (last part of the output, as also mentioned in the PR details, https://tex.stackexchange.com/a/314079).

This is not causing any issue other than tinytex attempting the installation twice: The LaTeX packages are installed correctly and can be successfully used on rocker/verse:latest and earlier, as my test above shows (https://github.com/rocker-org/rocker-versioned/pull/171#issuecomment-552825592)

So if there are actual issues with using TeX Live 2016 rocker/verse:latest when rendering R Markdown to PDF, they should be looked for elsewhere I believe. Happy to investigate further if you have any pointer.

We clearly have an issue rocker/verse:develop, although we would not be in a betters position if we would not be using a version-stable snapshot: The latest TeX Live update now requires an installation of the rmarkdown package from GitHub (https://github.com/yihui/tinytex/issues/152#issuecomment-552501131)

riccardoporreca commented 4 years ago

@cboettig

I believe this is what is now biting @annakrystalli in https://github.com/rocker-org/rocker-versioned/issues/172. I'm not sure of a great fix -- we can add rstudio as the owner of /opt/TinyTex but it will still cause things to break for other users... Thoughts?

I also had a quick look at the Dockerfile. There however the base image rocker/binder:3.6.0 does not seem to rely on / being rebuilt after making TinyTeX version-stable in rocker/verse

docker pull rocker/binder:3.6.0
docker run --rm rocker/binder:3.6.0 sh -c ' \
  cat /etc/os-release | grep PRETTY_NAME \
  && echo repo: $CTAN_REPO \
  && tex --version | head -n 1 \
'

PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
repo:
TeX 3.14159265 (TeX Live 2019)

cboettig commented 4 years ago

Thanks for this! I was worried the successful test was due only to bbm already being installed, but I do see it is being installed twice. that explains why everything I tested gave me the 'already installed' message!

So, with the 2018 version in devel, it does look like we have a permissions issue we will have to work around.

yeah, good point about the binder case, tags older than latest don't get automatically rebuilt, so the changes to verse:3.6.0 haven't propagated to binder:3.6.0. Does seem to be a permissions issue there too though

riccardoporreca commented 4 years ago

@cboettig, I have been investigating the issue with the frozen TeX Live 2018 snapshot we use in rocker/verse:devel, which I was wrongly assuming to include the fix discussed in https://github.com/yihui/tinytex/issues/77#issuecomment-466584510. Thanks to the great feedback from @norbusan I got a lot of insight into the issue and we are following up in the comments to his blog post.

Update: The approach discussed with @norbusan allows to manually "patch" the TeX Live installation to include the fix discussed in https://github.com/yihui/tinytex/issues/77#issuecomment-466584510, which is available in telnet archive but only used for the installer. I will include this in the rocker/verse:devel as part of a new PR.

cboettig commented 4 years ago

@riccardoporreca Re your updated comment;

The approach discussed with @norbusan allows to manually "patch" the TeX Live installation to include the fix discussed in yihui/tinytex#77 (comment), which is available in telnet archive but only used for the installer. I will include this in the rocker/verse:devel as part of a new PR.

Is it possible to take this approach to patch what looks like to be mistaken error messages in the 2016 version in latest images when tlmgr installs? While these errors seem to be harmless, if the user runs installs in a shell session with error handling set by set -e, this can cause the build to fail, as @Midnighter reports doing in https://github.com/rocker-org/rocker/issues/370#issuecomment-562853409

Appreciate your thoughts. Thanks!

riccardoporreca commented 4 years ago

Is it possible to take this approach to patch what looks like to be mistaken error messages in the 2016 version in latest images when tlmgr installs?

Unfortunately the fix to TeX Live mentioned by @norbusan in https://tex.stackexchange.com/a/314079 came after TeX Live 2016 was frozen, hence the same approach does not apply here.

However, the corresponding fix was pretty straight-forward and isolated, see https://git.texlive.info/texlive/commit/Master/tlpkg/TeXLive/TLUtils.pm?id=69cee5e1ce4b20f6ebb6af77e19d49706a842a3e, so we could back-port it to the version-stable frozen TeX Live 2016 we use in stretch-based rocker/verse images.

Here is a quick PoC, reproducing the error...

docker run --rm rocker/verse:3.6.1 bash -c '
tlmgr install multirow \
&& tlmgr path add \
&& echo SUCCESS' 
## [...]
## [1/1, ??:??/??:??] install: multirow [3k]
## [...]
## tlmgr: An error has occurred. See above messages. Exiting.

...and w/o error after applying the patch

docker run --rm rocker/verse:3.6.1 bash -c '
apt-get update && apt-get install patch \
&& wget -qO- \
   "https://git.texlive.info/texlive/patch/Master/tlpkg/TeXLive/TLUtils.pm?id=69cee5e1ce4b20f6ebb6af77e19d49706a842a3e" | \
   patch -i - /opt/TinyTeX/tlpkg/TeXLive/TLUtils.pm \
&& apt-get purge -y patch && apt-get clean \
&& tlmgr install multirow \
&& tlmgr path add \
&& echo SUCCESS' 
## [...]
## The following NEW packages will be installed:
##   patch
## [...]
## patching file /opt/TinyTeX/tlpkg/TeXLive/TLUtils.pm
## Hunk #1 succeeded at 1896 (offset -30 lines).
## [...]
## Removing patch (2.7.5-1+deb9u2) ...
## [...]
## [1/1, ??:??/??:??] install: multirow [3k]
## [...]
## SUCCESS

@cboettig, if you agree with this approach I am happy to contribute a PR bringing this patch to all the rocker/verse Dockerfiles. Update: Draft PR #184 to make things easier to assess and to get further feedback.

cboettig commented 4 years ago

Thanks @riccardoporreca this looks reasonable to me.

rocker-org / rocker-versioned

Version-stable CTAN repo for the TinyTeX install #171