arXiv / arxiv-submission-ui

User interface of NG submit system.
MIT License
2 stars 6 forks source link

ARXIVNG-2787: Reduce size of converter image (Create image for each tree) #168

Closed DavidLFielding closed 4 years ago

DavidLFielding commented 4 years ago

The initial goal of ticket ARXIVNG-2787 was to create a reduced size converter image but due to the fact that each TeX Live tree is so huge it was not possible to drastically reduce the size of the current converter image (with all trees).

I ended up creating a new image for each individual TeX tree. In reality the teTeX 2 tree uses three different local texmf trees and w2c binaries over a period of a few years so I kept these together.

While the README.md instructed users to add the CONVERTER_DOCKER_IMAGE setting to choose the TeX tree the docker-compose YAML was not reading it. I fixed this so now changing converters is a matter of changing the environment variable and restarting the application.

If you set the CONVERTER_DOCKER_IMAGE to a valid image the compiler service will download it for you. The other option is to download the image before starting the application.

There are several new images in the AWS ECR repository. The image with all trees still exists as the plain arxiv/converter image.

Another PR will cover changes to converter code to improve TeX tree selection when the image does not contain all of the trees or when an appropriate date is not specified.

SBBCornell commented 4 years ago

Just one thought: Maybe the warning messages after processing, with all caps NOT FOUND, might be a little confusing? Below the 2009 converter image was specified, so technically not finding the other compilers isn't an error.

[verbose]: Check: /opt_arxiv/arXivTeX/texlive/2016 [verbose]: Warning: arXiv TeX Live 2016 tree NOT FOUND [verbose]: Warning: arXiv TeX Live 2011 tree NOT FOUND [verbose]: Warning: TeX tree active on '29 Jan 2020' is not available. [verbose]: Using TeX Live 2009

Though the real use case of that scenario might be so infrequent it's not worth editing the error messages.

DavidLFielding commented 4 years ago

I'll merge this later today to give folks a chance to look at it. Most of the actual changes are in the arxiv-converter PR.