cli via docker does not edit the local config file

[dariober]

When running the cli via docker we need to check were it reads and writes the config file. Currently, this:

docker run apollo-cli config address http://localhost:1234

yarn run v1.22.19
warning package.json: No license field
$ tsx bin/dev.js config address http://localhost:1234
"default.accessType" is required
Done in 0.75s.

Completes fine, but the local configuration file is not changed

[garrettjstevens]

We'll need to use a bind mount or a volume, probably. With a bind mount, I think it would look like -v "~/.config:/path/to/container/.config, where we'd have to figure out where the .config directory for the default user is in the container. This would make it so any changes in .config in the container would also happen in .config.

I think volumes are supposed to be a safer way of doing something similar, but I'm not as familiar as them. Basically you create a volume with a name, and then you can mount that volume every time the container is run and persist data that way without having to make any potentially conflicting changes to your local .config directory.

[dariober]

This Dockerfile, which is suspiciously simple, works...:

FROM node:18-alpine3.19
WORKDIR /app
RUN npm install -g @apollo-annotation/apollo-cli

... but usage seems a bit laborious.

First build a container named apollo-cnt:

docker build -t apollo-cnt .

To use Apollo we need to know where Apollo writes the config file in the container. With the cli in main (not the one on npm), you should be able to get it with:

docker run apollo-cnt apollo config --get-config-file
/root/.config/apollo-cli/config.yaml # <- You should get this

To configure Apollo for, e.g., root access and to login use:

docker run --network host -v ~/.config/apollo-cli:/root/.config/apollo-cli apollo-cnt apollo config address http://localhost:3999
docker run --network host -v ~/.config/apollo-cli:/root/.config/apollo-cli apollo-cnt apollo config accessType root
docker run --network host -v ~/.config/apollo-cli:/root/.config/apollo-cli apollo-cnt apollo config rootCredentials.username admin
docker run --network host -v ~/.config/apollo-cli:/root/.config/apollo-cli apollo-cnt apollo config rootCredentials.password pass
docker run --network host -v ~/.config/apollo-cli:/root/.config/apollo-cli apollo-cnt apollo login

~/.config/apollo-cli can be a directory of user's choice. If the config file does not exist on the localhost apollo/docker will create it but it will have restricted permissions.

To upload files we need to mount the relevant local directory (not tested because the npm version has only the cli for config and login). For example if you want to add an assembly from ~/Downloads/genome.fa, use:

docker run --network host \
| -v ~/.config/apollo-cli:/root/.config/apollo-cli \
| -v ~/Downloads:/host apollo-cnt \
| apollo assembly add-fasta -i /host/genome.fa

---

(I'll update the cli on npm to the one in the main branch)

[dariober]

In branch npm-update I updated the main branch to be publishable on npm and devel version 0.1.1 is now here.

Update and build container. Dockerfile:

FROM node:18-alpine3.19
WORKDIR /app
RUN yarn global add @apollo-annotation/apollo-cli@devel

docker build -t apollo-cnt .

docker run apollo-cnt apollo --version
@apollo-annotation/apollo-cli/0.1.1 linux-x64 node-v18.20.2

docker run apollo-cnt apollo config --get-config-file

 ›   ModuleLoadError: [MODULE_NOT_FOUND] import() failed to load 
 ›   /usr/local/share/.config/yarn/global/node_modules/@apollo-annotation/apoll
 ›   o-cli/dist/commands/config.js: Cannot find package 'undici' imported from 
 ›   /usr/local/share/.config/yarn/global/node_modules/@apollo-annotation/apoll
 ›   o-cli/dist/commands/config.js
 ›   Code: MODULE_NOT_FOUND

[garrettjstevens]

In packages/apollo-cli/package.json, undici should be in "dependencies", not "devDependencies". That should fix the error.

[garrettjstevens]

I think the simple Dockerfile is fine, it's always going to take a bit more configuration to use the CLI with docker instead of installing it directly. Also, the volume mounts you specify seem reasonable to me, it seems similar to usage of other tools with docker.

I think you can get rid of the WORKDIR, and possibly it would be good to add

ENTRYPOINT ["apollo"]

That allows you to leave off the apollo part of the command, so you can run something like:

docker run apollo-cli config --get-config-file

[garrettjstevens]

See the official AWS CLI Docker image that suggest commands pretty similar to the ones you described: https://hub.docker.com/r/amazon/aws-cli

[dariober]

In packages/apollo-cli/package.json, undici should be in "dependencies", not "devDependencies". That should fix the error.

This is included in 0.1.2 from https://github.com/GMOD/Apollo3/tree/npm-update.

---

In 122f8d2 I edited the Dockerfile to resolve the permission issues. I included some tests that pass locally, but fail on github workflow.

Run python3 ./test/test_docker.py
E
======================================================================
docker build --build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g) -t apollo .
ERROR: setUpModule (__main__)
docker run --network host -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/tmpTestDocker/.config:/home/apolloUser/.config/apollo-cli -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/test_data:/data apollo config address http://localhost:3999
docker run --network host -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/tmpTestDocker/.config:/home/apolloUser/.config/apollo-cli -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/test_data:/data apollo config accessType root
docker run --network host -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/tmpTestDocker/.config:/home/apolloUser/.config/apollo-cli -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/test_data:/data apollo config rootCredentials.username admin
docker run --network host -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/tmpTestDocker/.config:/home/apolloUser/.config/apollo-cli -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/test_data:/data apollo config rootCredentials.password pass
docker run --network host -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/tmpTestDocker/.config:/home/apolloUser/.config/apollo-cli -v /home/runner/work/Apollo3/Apollo3/packages/apollo-cli/test_data:/data apollo login
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/runner/work/Apollo3/Apollo3/packages/apollo-cli/./test/test_docker.py", line 26, in setUpModule
    shell(f"{apollo} login", timeout=60)
  File "/home/runner/work/Apollo3/Apollo3/packages/apollo-cli/test/utils.py", line 27, in __init__
    raise subprocess.SubprocessError(
subprocess.SubprocessError: 
STDOUT:

STDERR:
TypeError: fetch failed
    at fetch (/home/apolloUser/.config/yarn/global/node_modules/undici/index.js:112:13)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Login.startRootLogin (file:///home/apolloUser/.config/yarn/global/node_modules/@apollo-annotation/apollo-cli/dist/commands/login.js:132:26)
    at async Login.run (file:///home/apolloUser/.config/yarn/global/node_modules/@apollo-annotation/apollo-cli/dist/commands/login.js:89:35)
    at async Login._run (/home/apolloUser/.config/yarn/global/node_modules/@oclif/core/lib/command.js:311:22)
    at async Config.runCommand (/home/apolloUser/.config/yarn/global/node_modules/@oclif/core/lib/config/config.js:433:25)
    at async run (/home/apolloUser/.config/yarn/global/node_modules/@oclif/core/lib/main.js:92:16)
    at async main (file:///home/apolloUser/.config/yarn/global/node_modules/@apollo-annotation/apollo-cli/bin/run.js:5:3)
    at async file:///home/apolloUser/.config/yarn/global/node_modules/@apollo-annotation/apollo-cli/bin/run.js:8:1

EXIT CODE: 1

---

Maybe unrelated, on a virtual machine, the Dockerfile fails with:

docker build --build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g) -t apollo .

[+] Building 1.0s (5/7)                                                                                                                                                                                                                                                                                                                                              docker:default
 => [internal] load build definition from Dockerfile                                                                                                                                                                                                                                                                                                                           0.0s
 => => transferring dockerfile: 324B                                                                                                                                                                                                                                                                                                                                           0.0s
 => [internal] load metadata for docker.io/library/node:18-alpine3.19                                                                                                                                                                                                                                                                                                          0.8s
 => [internal] load .dockerignore                                                                                                                                                                                                                                                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                                                                                                                                                                                                                                                0.0s
 => CACHED [1/4] FROM docker.io/library/node:18-alpine3.19@sha256:4837c2ac8998cf172f5892fb45f229c328e4824c43c8506f8ba9c7996d702430                                                                                                                                                                                                                                             0.0s
 => ERROR [2/4] RUN addgroup -g 1000 apolloUser                                                                                                                                                                                                                                                                                                                                0.2s
------                                                                                                                                                                                                                                                                                                                                                                              
 > [2/4] RUN addgroup -g 1000 apolloUser:
0.150 addgroup: gid '1000' in use
------
Dockerfile:6
--------------------
   4 |     ARG GROUP_ID
   5 |     
   6 | >>> RUN addgroup -g $GROUP_ID apolloUser
   7 |     RUN adduser -u $USER_ID -G apolloUser -g "" -D apolloUser
   8 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c addgroup -g $GROUP_ID apolloUser" did not complete successfully: exit code: 1

(I'll investigate...)

[garrettjstevens]

In 122f8d2 I edited the Dockerfile to resolve the permission issues.

I don't think this will work because the user when the image is build from the Dockerfile is built is not necessarily the same user that runs a container with that image. e.g. if we publish the image, the users will not have the same user ID as the GitHub action the image is built on.

I can see two options for the permissions issue you describe:

1. Have the user pass in their user and group to docker run with the --user flag
2. Enforce that the config file must already exist so docker run doesn't create one as its root user

I favor solution 2, since changing the user that's used inside the container can have other side effects.

If we go with solution 2, I propose adding a check in the CLI code before the config file is created that looks for an environment variable APOLLO_DISABLE_CONFIG_CREATE, and if that env variable is some truthy value, exit with an error instead of creating the config file. We can then set that env variable to "true" in the ENTRYPOINT of the Dockerfile, and it won't affect users using it outside of Docker.

[dariober]

Enforce that the config file must already exist so docker run doesn't create one as its root user

I agree this is better than option 1. It should be done in d4208290696033d4baadf04f6819078ac9ba415b

[dariober]

Commit 7eb9b61 has script makeGitTag.py to update the package versions, create a new tag, and push to github.

Workflow publish.yml publishes the new tag release to npm and github packages. The code for build-and-push-docker moved to build-and-push-docker.yml.

It seems to work, tested with:

./makeGitTag.py -t v0.1.5 -m 'Test pubblication 2'