Cannot launch on NCAR's casper

[Yefee]

Hi all,

Thanks for the wonderful work. I am trying to install the docker image using apptainer on NCAR's casper supercomputer. I installed it successfully, but I cannot launch it. Below is my procedure. Any suggestions would be appreciated!

The thredds sever is installed.

INFO:    Environment variable SINGULARITY_TMPDIR is set, but APPTAINER_TMPDIR is preferred
INFO:    Starting build...
Getting image source signatures
Copying blob 94713d002d7b skipped: already exists
Copying blob cfcf356fa6e6 skipped: already exists
Copying blob e8c0f6315078 skipped: already exists
Copying blob e07cd336c68a skipped: already exists
Copying blob e6cc498ca00f skipped: already exists
Copying blob 23828d760c7b skipped: already exists
Copying blob 02fe846b80d2 skipped: already exists
Copying blob 67a48b508d87 skipped: already exists
Copying blob a2b1ca3d0a24 skipped: already exists
Copying blob 4235b50a2db5 skipped: already exists
Copying blob 4dfa983196a4 skipped: already exists
Copying blob 7c6ecb0c1504 skipped: already exists
Copying blob 43c36ce413c1 skipped: already exists
Copying blob 0edd5aac7924 skipped: already exists
Copying blob 8e794e46c489 skipped: already exists
Copying blob a51f283b3436 skipped: already exists
Copying blob c08e59a0e5bd skipped: already exists
Copying blob 9d8eb9c25daa skipped: already exists
Copying blob 4f4fb700ef54 skipped: already exists
Copying config e2759ed79a done
Writing manifest to image destination
Storing signatures
2024/04/04 16:20:10  info unpack layer: sha256:23828d760c7b04df02891af556c40ca44c2dd79d6837ea6f18fac24f4108448c
2024/04/04 16:20:20  info unpack layer: sha256:cfcf356fa6e6902e661a53c29cf3b351f284090cb2c0de2ad6c5c238a6fdb9a8
2024/04/04 16:20:30  info unpack layer: sha256:e8c0f63150781234e1d33ffed6a942ef24e0e4c5129bfebcd44aee4f26e06a5d
2024/04/04 16:20:33  info unpack layer: sha256:e07cd336c68a38b22695a291ab59bfb390303d2092883bc249f2c3343f8433d5
2024/04/04 16:20:33  info unpack layer: sha256:e6cc498ca00faabc0410d3bbc08149d8cf04663aebf55f88aaa7ea7e46bea8c6
2024/04/04 16:20:33  info unpack layer: sha256:94713d002d7bfa272616a2497571bf133537aa84533fdb80c6c9f6039cf6ec3f
2024/04/04 16:20:33  info unpack layer: sha256:67a48b508d87ed4c854209a75341d3245c7a3dd0163dbf4b45045b79223b5f49
2024/04/04 16:20:36  info unpack layer: sha256:4235b50a2db537ec0c2ca866bee6b0019c0c5457e3f69fd7c9963840775418e5
2024/04/04 16:20:36  info unpack layer: sha256:02fe846b80d2cf9545c530c522c7d20b5b66b8c11212011d42a4eaa691737844
2024/04/04 16:20:36  info unpack layer: sha256:a2b1ca3d0a24ef0ae81b4259e1a9ab5c6fe1665c5a152f227d9239377ad6fe54
2024/04/04 16:20:36  info unpack layer: sha256:7c6ecb0c1504f503b95dae405d9b9285407697cd7d98ccc95fb76457c0a1ed84
2024/04/04 16:20:36  info unpack layer: sha256:4dfa983196a4c1be05134014ce061693fcb68fd1d355965698e4e8502f1f91ac
2024/04/04 16:20:36  info unpack layer: sha256:43c36ce413c19198f3d2eb8c55c9611447c6211c955535420475059d8f2f2cf9
2024/04/04 16:20:36  info unpack layer: sha256:0edd5aac7924f757cd2083163316832dcad1529341477ada4207b78d5cf5234c
2024/04/04 16:20:37  info unpack layer: sha256:8e794e46c489313b759a75c1767ef808c253299b545f5875f75c2801c7579759
2024/04/04 16:20:37  info unpack layer: sha256:a51f283b3436e533cddffe741e8953f83938c32fbdf71bf3d1b028ef15419c43
2024/04/04 16:20:37  info unpack layer: sha256:c08e59a0e5bd19a268ae8e7b11083e63622dcb6065e683d13ad5e802a49d5b52
2024/04/04 16:20:37  info unpack layer: sha256:9d8eb9c25daa6a5535d22ca068dd15f132a5dc421d6cbf9dcc9c2ff24c44924c
2024/04/04 16:21:06  info unpack layer: sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
INFO:    Creating SIF file...
INFO:    Build complete: tds.sif

I cannot launch it.

apptainer run --bind $(pwd)/content:/data tds.sif
INFO:    Environment variable SINGULARITY_TMPDIR is set, but APPTAINER_TMPDIR is preferred
INFO:    fuse2fs not found, will not be able to mount EXT3 filesystems
groupadd: GID '1000' already exists
chown: invalid user: ‘tomcat:tomcat’
error: failed switching to "tomcat": unable to find user tomcat: no matching entries in passwd file

[ana-v-espinoza]

CC: @julienchastang

Hello Yefee,

Thank you for your interest in thredds-docker!

Apologies for the late response, I was traveling yesterday and just now have gotten a chance to respond. It looks like you're running the image using apptainer/singularity, which we don't support so we can't guarantee any compatibility with it. However, I believe your issue would be related to this line in the container's entrypoint.sh which it inherits from unidata/tomcat-docker.

This portion of the entrypoint is meant to allow for a consistent UID both within the container and on the host machine. I'm not particularly familiar with the ins and outs of apptainer/singularity, but I do know that it allows for you (i.e., a user of an HPC system) to launch containers without elevated privileges and thus has some of this user management related things built in.

I imagine that your problem is due to this difference between docker and singularity.

As I mentioned, however, we don't support singularity, so I'm not sure if there's anything that can be easily done to fix this.

Are you attempting to run on casper for a particular reason, or is this just for testing purposes? It's my understanding that HPC centers typically don't allow for long-running jobs, especially those that may require opening ports, such as THREDDS.

I would recommend that, if possible, to acquire access to a system where you can run Docker containers as opposed to having to rely on apptainer/singularity.

I hope this information was useful.

I will leave this issue open until next Wednesday in case you need anything else. Let me know if there's another way I can help.

Best, Ana V. Espinoza

[Yefee]

Dear Ana,

Thanks for your detailed response.

However, I believe your issue would be related to this line in the container's entrypoint.sh which it inherits from unidata/tomcat-docker.

Would it be possible to manually provide an UID for the application? I believe I could use my own UID?

Are you attempting to run on casper for a particular reason, or is this just for testing purposes? It's my understanding that HPC centers typically don't allow for long-running jobs, especially those that may require opening ports, such as THREDDS.

I am using Casper to read a ncml CMIP6 data file. This is the repo I am trying to make it work. It requires to initiate an TREDDS server to read the data via OpenDap. There would be no problem with opening ports.

Thanks! Chengfei

[julienchastang]

For configuring UID/GID see here.