During my work in Google Cloud I discovered that the submission agent and worker containers were exiting during the initialization process. This PR addresses several issues that were causing problems in addition to adding significant debug level logging to help me better understand what is going on in the code.
The best way to test this PR is to check out this branch ARXIVNG-4062 and the accompanying arxiv-submission-core branch ARXIVNG-4157, build the agent/work containers [core], and docker-compose pull/build/up in the Submission UI branch [subui].
The main idea is that the agent/worker containers no longer crash. When you submit a paper you should see the agent launching numerous actions on the submission.
After you check out the arxiv-submission-core branch you will need to build the agent/worker container. Submission UI's docker-compose.yml is currently expecting 0.1.15.
$ docker build . -t submission-agent:0.0.15
After you check out arxiv-submission-ui you will want to follow the instructions for setting environment variables and then run docker-compose pull/build/up. If you don't have a local compiler image you will want to either 1) set environment variable below along with ASW credentials and let Submission UI's Compiler service download the converter image, or 2) you can download it manually (there are instructions in the arxiv-submission-ui README.
After running 'docker-compose up' (or 'docker compose up' to use version included in docker toolkit) you will see a flurry of log messages. It takes a minute or two for things to start up and settle down.
If you want to watch the agent (or worker) logs you can do something like:
docker logs -f arxiv-submission-agent
The worker log level is currently at 40 so not much is displayed at the moment. Feel free to adjust this to 10 if you can't get enough logs.
You are now ready to run through the submission process. There is a directory of test submissions in the arxiv-filemanager repository under 'test_files_upload'. I use upload2.tar.gz, which is a working submission that successfully compiles.
During my work in Google Cloud I discovered that the submission agent and worker containers were exiting during the initialization process. This PR addresses several issues that were causing problems in addition to adding significant debug level logging to help me better understand what is going on in the code.
The best way to test this PR is to check out this branch ARXIVNG-4062 and the accompanying arxiv-submission-core branch ARXIVNG-4157, build the agent/work containers [core], and docker-compose pull/build/up in the Submission UI branch [subui].
The main idea is that the agent/worker containers no longer crash. When you submit a paper you should see the agent launching numerous actions on the submission.
After you check out the arxiv-submission-core branch you will need to build the agent/worker container. Submission UI's docker-compose.yml is currently expecting 0.1.15.
After you check out arxiv-submission-ui you will want to follow the instructions for setting environment variables and then run docker-compose pull/build/up. If you don't have a local compiler image you will want to either 1) set environment variable below along with ASW credentials and let Submission UI's Compiler service download the converter image, or 2) you can download it manually (there are instructions in the arxiv-submission-ui README.
CONVERTER_DOCKER_IMAGE=626657773168.dkr.ecr.us-east-1.amazonaws.com/arxiv/converter-2020:0.1.0
After running 'docker-compose up' (or 'docker compose up' to use version included in docker toolkit) you will see a flurry of log messages. It takes a minute or two for things to start up and settle down.
If you want to watch the agent (or worker) logs you can do something like:
docker logs -f arxiv-submission-agent
The worker log level is currently at 40 so not much is displayed at the moment. Feel free to adjust this to 10 if you can't get enough logs.
You are now ready to run through the submission process. There is a directory of test submissions in the arxiv-filemanager repository under 'test_files_upload'. I use upload2.tar.gz, which is a working submission that successfully compiles.