search

MacRemoteSensing / rstudio_container

Recipe to build a base docker container with RStudio Server, to be then run with apptainer in Alliance Can
MIT License
2 stars 0 forks source link

Issue in Apptainer #1

Closed ricardobarroslourenco closed 1 year ago

ricardobarroslourenco commented 1 year ago

When running in Docker, things are ok. However, once converted to Apptainer, there is an issue of crashing when starting. Some of the error:

(base) [lourenco@gra-login1 ~]$ salloc --time=0:30:0 --account=rrg-ggalex --cpus-per-task=8 --mail-user=barroslr@mcmaster.ca --mail-type=ALL
salloc: Pending job allocation 6618384
salloc: job 6618384 queued and waiting for resources
salloc: job 6618384 has been allocated resources
salloc: Granted job allocation 6618384
salloc: Waiting for resource configuration
salloc: Nodes gra106 are ready for job
(base) [lourenco@gra106 ~]$ module load apptainer
(base) [lourenco@gra106 ~]$ apptainer shell --nv -B /home -B /project -B /scratch ./projects/rrg-ggalex/lourenco/rstudio_container_rogers_v0_2.sif 
INFO:    underlay of /usr/bin/nvidia-smi required more than 50 (901) bind mounts
Apptainer> rstudio-server start --server-user=$(whoami)
TTY detected. Printing informational message about logging configuration. Logging configuration loaded from '/cvmfs/soft.computecanada.ca/gentoo/2020/etc/xdg:/etc/xdg/rstudio/logging.conf'. Logging to '/home/lourenco/.local/share/rstudio/log/rserver.log'.
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
Apptainer> ^C
Apptainer> rstudio-server start 
TTY detected. Printing informational message about logging configuration. Logging configuration loaded from '/cvmfs/soft.computecanada.ca/gentoo/2020/etc/xdg:/etc/xdg/rstudio/logging.conf'. Logging to '/home/lourenco/.local/share/rstudio/log/rserver.log'.
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down

Chance that is related to this issue ( https://github.com/grst/rstudio-server-conda/issues/16#issuecomment-935858216 , https://github.com/grst/rstudio-server-conda/pull/18 )

ricardobarroslourenco commented 1 year ago

Related: https://github.com/rocker-org/website/commit/8066ce83f0f11a95c0b3937aa86df409c06dc1ed

ricardobarroslourenco commented 1 year ago

More detail on build warnings, and issue when trying to run locally:

(base) ricardo@ricardo-XPS-8940:~/tmp/test_apptainer$ apptainer build rstudio_container_rogers_v0_2.sif docker://ricardobarroslourenco/rstudio_container:rogers_v0_2
WARNING: /usr/local/etc/singularity/ exists, migration to apptainer by system administrator is not complete
INFO:    Starting build...
Getting image source signatures
Copying blob dbf6a9befcde [--------------------------------------] 0.0b / 28.2MiB
Copying blob 217d63d1d724 [--------------------------------------] 0.0b / 1.8KiB
Copying blob 3708536563ca [--------------------------------------] 0.0b / 255.5MiB
Copying blob 2c6ea4ca9e69 [--------------------------------------] 0.0b / 22.8KiB
Copying blob 81acad82b1c8 [--------------------------------------] 0.0b / 26.3MiB
Copying blob 60f7d8b6ad61 [--------------------------------------] 0.0b / 248.9MiB
Copying blob ae0d911bd170 [--------------------------------------] 0.0b / 26.6KiB
Copying blob 277d92ae60fb [--------------------------------------] 0.0b / 53.6KiB
Copying blob d7255bd380f3 [--------------------------------------] 0.0b / 161.3MiB
Copying blob 359797b66c05 [--------------------------------------] 0.0b / 557.1MiB
Copying blob ef2a2702991e [--------------------------------------] 0.0b / 317.8MiB
Copying blob 4e550c7cbc52 [--------------------------------------] 0.0b / 26.7MiB
Copying blob ac8f8adeaf08 [--------------------------------------] 0.0b / 6.7MiB
Copying blob f836abd250ce [--------------------------------------] 0.0b / 23.8KiB
Copying blob a2573f04303f [--------------------------------------] 0.0b / 163.0b
Copying blob 118234d44966 [--------------------------------------] 0.0b / 378.8MiB
Copying blob 6fca68d8a19f [--------------------------------------] 0.0b / 153.0b
Copying config 29cdeb01fe done  
Writing manifest to image destination
Storing signatures
2023/05/19 15:21:42  info unpack layer: sha256:dbf6a9befcdeecbb8813406afbd62ce81394e3869d84599f19f941aa5c74f3d1
2023/05/19 15:21:43  info unpack layer: sha256:217d63d1d724ea13d64bdc0e502600f63768eccd042f56bead1ebbdeab1eea1f
2023/05/19 15:21:43  info unpack layer: sha256:3708536563ca0aa5e01d6a2a0442b71b5968440adae57e70760c70a9d881059c
2023/05/19 15:21:47  info unpack layer: sha256:2c6ea4ca9e69125a416120227c33adc8f9c9c327040da2d5f717eea89f2304ee
2023/05/19 15:21:47  info unpack layer: sha256:81acad82b1c8572113c08dd0ac27d36b1bc57a037236219bdeb72cb58a2a2488
2023/05/19 15:21:47  info unpack layer: sha256:60f7d8b6ad61c98a6a307f7c1a23cf42a43188cfa86450659e83256d0091b4f0
2023/05/19 15:21:52  info unpack layer: sha256:ae0d911bd1709684829b5c8663f9ca9be92cc5bf1e409821cc69e59414edbc73
2023/05/19 15:21:52  info unpack layer: sha256:277d92ae60fb3a3fb7877cf56fedc7d516477687738961e5d3091450c9404f6d
2023/05/19 15:21:52  info unpack layer: sha256:d7255bd380f3e2a804e0a06c66a9afc529b097a559f9360901e51905a986d74c
2023/05/19 15:21:54  info unpack layer: sha256:359797b66c054b7f024da26f6835725b5b3b7a6cb0f9c51e3125e129414e5db6
2023/05/19 15:21:54  warn rootless{opt/texlive/texmf-var/fonts/map/dvips/updmap/psfonts.map} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:21:54  warn rootless{opt/texlive/texmf-var/fonts/map/pdftex/updmap/pdftex.map} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/linux} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/context} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/context.lua} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/dvilualatex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/dviluatex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/dvipdfm} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/dvipdfmx} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/ebb} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/etex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/extractbb} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/fmtutil} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/fmtutil-sys} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/fmtutil-user} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/inimf} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/initex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/latex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/lualatex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/luaotfload-tool} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/man} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/mktexfmt} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/mktexlsr} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/mktexmf} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/mktexpk} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/mktextfm} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/mtxrun} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/mtxrun.lua} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/pdfcrop} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/pdfetex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/pdflatex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/rpdfcrop} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/rungs} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/simpdftex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/texhash} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/texlua} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/texluac} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/tlmgr} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/updmap} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/updmap-sys} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/updmap-user} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/xelatex} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/xelatex-unsafe} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:00  warn rootless{usr/local/texlive/bin/x86_64-linux/xetex-unsafe} ignoring (usually) harmless EPERM on setxattr "user.rootlesscontainers"
2023/05/19 15:22:02  info unpack layer: sha256:ef2a2702991ec1472729585b4b65df9eb8c69579ba96cad67f3a48b37ba5f123
2023/05/19 15:22:08  info unpack layer: sha256:4e550c7cbc52e857c8749f4aac9d646aa131f376a0c3feba675bff0d4bb06bad
2023/05/19 15:22:09  info unpack layer: sha256:ac8f8adeaf08b956f798f573b28872c5b4d44f7a06220d17781bd1145ae83a0c
2023/05/19 15:22:09  info unpack layer: sha256:f836abd250ce15a8674ea4a611e5425294d10ab9199fc78bc1c5a6211ec8b944
2023/05/19 15:22:09  info unpack layer: sha256:a2573f04303fe7f6cac623e5658df0303fd4b5d41349d70d79b72c97d967622f
2023/05/19 15:22:09  info unpack layer: sha256:118234d44966f5b9e85926ee56867dab53fe4d2dfd36bd0f9cf8faa56ab165aa
2023/05/19 15:22:15  info unpack layer: sha256:6fca68d8a19f5daab716d2d82ff6e6e9a5afc83520d8a77abce3ada7430d0439
WARNING: /usr/local/etc/singularity/ exists, migration to apptainer by system administrator is not complete
INFO:    Creating SIF file...
INFO:    Build complete: rstudio_container_rogers_v0_2.sif
(base) ricardo@ricardo-XPS-8940:~/tmp/test_apptainer$ apptainer shell --nv -B /home -B /project -B /scratch rstudio_container_rogers_v0_2.sif 
WARNING: /usr/local/etc/singularity/ exists, migration to apptainer by system administrator is not complete
WARNING: skipping mount of /project: stat /project: no such file or directory
WARNING: skipping mount of /scratch: stat /scratch: no such file or directory
FATAL:   container creation failed: mount /project->/project error: while mounting /project: mount source /project doesn't exist
(base) ricardo@ricardo-XPS-8940:~/tmp/test_apptainer$ apptainer shell --nv -B /home rstudio_container_rogers_v0_2.sif 
WARNING: /usr/local/etc/singularity/ exists, migration to apptainer by system administrator is not complete
Apptainer> rstudio-server start
TTY detected. Printing informational message about logging configuration. Logging configuration loaded from '/etc/rstudio/logging.conf'. Logging to 'syslog'.
System has not been booted with systemd as init system (PID 1). Can't operate.
Failed to connect to bus: Host is down
ricardobarroslourenco commented 1 year ago

Rebuilding the image with sudo, removes the build warnings, but it does not impact the error reported.

ricardobarroslourenco commented 1 year ago

@e-kotov I wonder if the error I am getting here is related to the way I am launching RStudio server (as mentioned in the instructions here https://github.com/rocker-org/website/commit/8066ce83f0f11a95c0b3937aa86df409c06dc1ed ). Can you give me a hint?

ricardobarroslourenco commented 1 year ago

Refer to @vsoch comment : https://twitter.com/vsoch/status/1659668102328778752?cxt=HHwWgIDUxZDWqYguAAAA .

tomgreen66 commented 1 year ago

If I understood the question - digging out a script I wrote the following to launch with Singularity. MYPORT being free port to listen on and SVRDATADIR being a temp directory and COOKIE_KEY being a file inside SVRDATADIR.

PASSWORD='$PASSWORD' RSTUDIO_SESSION_TIMEOUT='0' /usr/lib/rstudio-server/bin/rserver --auth-none=0  --auth-pam-helper-path=pam-helper --www-address=0.0.0.0 --www-port=$MYPORT --server-data-dir=$SVRDATADIR --secure-cookie-key-file=$COOKIE_KEY --auth-encrypt-password=0 --server-set-umask=0 --server-user=$USER

Also set some bindings where RSTUDIO_DB just some temporary location and MYRSTUDUIO_LIB was a unique location to store Rstudio install files not to clash with locally installed R. Cgroup needed to get rid of some errors when running.

export SINGULARITY_BIND=$RSTUDIO_DB:/var/lib/rstudio-server,$MYRSTUDIO_LIB:$MYR_LIB,/sys/fs/cgroup

Have a complete script if this looks helpful.

e-kotov commented 1 year ago

@ricardobarroslourenco if you follow https://rocker-project.org/use/singularity.html this should work for you:

Build:

apptainer build rstudio_container_rogers_v0_2.sif docker://ricardobarroslourenco/rstudio_container:rogers_v0_2

Prepare folders to bind to the container (as Apptainer is different from Docker and requires these folders for rocker images to run):

mkdir -p run var-lib-rstudio-server .rstudio
printf 'provider=sqlite\ndirectory=/var/lib/rstudio-server\n' > database.conf

Run (I omitted your bind folders, but you can add them back, also, I selected a random non-standard port, which again can be anything):

apptainer exec --env PASSWORD='set_your_password' --bind run:/run,var-lib-rstudio-server:/var/lib/rstudio-server,database.conf:/etc/rstudio/database.conf,.rstudio:/home/rstudio/.rstudio/ rstudio_container_rogers_v0_2.sif /usr/lib/rstudio-server/bin/rserver --auth-none=0 --auth-pam-helper-path=pam-helper --server-user=$(whoami) --www-port=9102
ricardobarroslourenco commented 1 year ago

Thanks @e-kotov for the clarification. It is working like a charm now.