Closed tbrittoborges closed 8 months ago
Hi @HaraldWilhelmi,
can you test what we discussed earlier today?
The app leaves in the shiny server: (10.250.140.12).
The docker file is at /home/tbrittoborges/apps/nmd_transcriptome/Dockerfile
Thanks!
The present version of the app is based on the the Docker image rocker/verse:4.2.1. That in turn uses an old Ubuntu version (focal), for which no criu is available. So as a first step we moved to a private repo (git@github.com:dieterich-lab/nmdtx.git) and try to upgrade to a much newer image/R version (4.3.1). That will give us a much newer Ubuntu version, for which we have at least a pre-compiled PPA version.
Using this instructions to upgrade renv.lock: https://inbo.github.io/tutorials/tutorials/r_renv_update/
Had to do manually:
R -e 'renv::install("dzhang32/ggtranscript")'
Seems that it is easier to stay with the old software versions. The key seems to be to get criu from a PPA repo:
add-apt-repository ppa:criu/ppa
See here for the app initialization profile. profile.html.zip
Status from Monday: We have a working implementation with CRIU now - see https://github.com/dieterich-lab/nmdtx/tree/main/deploy. With naive approach to just preload 'library('nmdtx') we just get an improvement of 2 seconds (13 s -> 11 s).
Profiling the app within the container is not trivial. Here are my steps. As of now, the profiler is not outputting the report as expected.
Connect to the shiny-new server:
ssh tbrittoborges@10.250.140.14
cd /home/tbrittoborges/nmdtx/deploy
Change rstart.R
by adding the preload command, in the example the command is
commented out:
# library("nmdtx")
while (! file.exists('/restore.marker')) {
print ('waiting....')
print('Library loaded')
Sys.sleep(0.5)
}
system(paste('mkdir -p', tempdir()))
system('tail --follow=name /criu_dumps/rlogs.log > /criu_dumps/rlogs_container.log &')
shiny_user <- basename(readLines('/shiny_user.txt'));
shiny_groups <- basename(readLines('/shiny_usergroups.txt'));
print(paste('User', shiny_user))
Sys.setenv('SHINYPROXY_USERNAME' = shiny_user, 'SHINYPROXY_USERGROUPS' = shiny_groups)
print(Sys.getenv('SHINYPROXY_USERNAME'))
print(Sys.getenv('SHINYPROXY_USERGROUPS'))
options('shiny.port'=3838,shiny.host='0.0.0.0')
p <- profvis::profvis({
print(nmdtx::run_app())
})
htmlwidgets::saveWidget(p, "profile.html")
Build the image:
./build.sh
Run the container on the detached mode, so we can copy the profviz output:
docker run --privileged -it --entrypoint '/restore.sh' --detach nmdtx
Keep track of the init.
docker logs -f $(docker ps -q | head -n1)
Once the app is done, copy the profile file from the container to the server.
docker cp $(docker ps -q | head -n1):profile.html profile.html
Copy the file from a remote to your local machine, give it a descriptive name.
scp tbrittoborges@shinynew:/nmdtx/deploy/profile.html profile_noChange.html
Once done, stop the container, and profile the next initialization strategy.
docker stop $(docker ps -q | head -n1)
Important if the following message repeats after ./build.sh
, there is
something wrong rstart.R
:
ls: cannot access '/criu_dumps/dump.done': No such file or directory
Waiting for dump...
About:
ls: cannot access '/criu_dumps/dump.done': No such file or directory Waiting for dump...
That comes from the loop, which check if the dump is done. It is normal for the start, but should stop after 2 or 3 interations.
Yes, that is fine. The problem is at the profiling line:
p <- profvis::profvis({
print(nmdtx::run_app())
})
The profvis function requires a stop signal, and I could not find a way to do that within the docker environment.
For documentation, I tried the solution for remote server, from here: https://rstudio.github.io/profvis/faq.html#can-i-profile-a-shiny-application-running-on-a-remote-server
But current the app has not stop, but I could add one.
Rprof(strftime(Sys.time(), "%Y-%m-%d-%H-%M-%S.Rprof"),
interval = 0.01, line.profiling = TRUE,
gc.profiling = TRUE, memory.profiling = TRUE)
onStop(function() {
Rprof(NULL)
})
https://support.openanalytics.eu/t/create-running-user-proxies-containers-in-advance/2521/2