Problem when running through docker

[fvalle1]

I followed the instruction for a docker installation, then I set LWREF (/home/user/Developer/LOLAweb/) and LWLOCAL (/home/user/Developer/LOLAweb/data/) environment variables.

I run a container

docker run -d -p 80:80 -e LWREF=$LWREF -e LWLOCAL=$LWLOCAL --volume ${LWLOCAL}:${LWLOCAL} --volume ${LWREF}:${LWREF} --volume ${LWLOCAL}/shinylog:/var/log/shiny-server --name lola databio/lolaweb:latest

I go to the website, when I upload a file (or an example .bed file) the web server crashes.

On the log I found

Calculating region set enrichments ... Loading example data Reading collection annotations: codex: found collection annotation:/home/user/Developer/LOLAweb/databases//Core/mm10/codex/collection.txt Timing stopped at: 0.138 0 0.15 Warning: Error in fread: file not found: /home/user/Developer/LOLAweb/databases//Core/mm10/codex/collection.txt 122: fread 121: readCollectionAnnotation 120: loadRegionDB 118: eventReactiveHandler [/srv/shiny-server/LOLAweb/apps/LOLAweb/app.R#514] 74: rawdat_nocache 72: observeEventHandler [/srv/shiny-server/LOLAweb/apps/LOLAweb/app.R#387] 1: runApp Warning: Error in fread: file not found: /home/user/Developer/LOLAweb/databases//Core/mm10/codex/collection.txt 122: fread 121: readCollectionAnnotation 120: loadRegionDB 118: eventReactiveHandler [/srv/shiny-server/LOLAweb/apps/LOLAweb/app.R#514] 74: rawdat_nocache 72: observeEventHandler [/srv/shiny-server/LOLAweb/apps/LOLAweb/app.R#387] 1: runApp

What could be the problem? /home/user/Developer/LOLAweb/databases//Core/mm10/codex/collection.txt exists on the running container, I am able to read it correctly..

[vpnagraj]

hi @fvalle1 thanks for raising this issue to our attention

i've identified some gaps in our docker documentation.

while we address those gaps, could you please try the following and let us know if it works on your end? if so you should be able to adapt to work with the data you're using for your customized local LOLAweb instance

## assign env vars for data path
## NOTE: must include trailing /
LWLOCAL="/path/to/local/data/"
LWREF="/path/to/local/data/"

## change reference data dir
cd $LWREF

## create dir for databases
mkdir -p databases
## create examples and universe dir
## NOTE: these must include subdirs named corresponding to appropriate ref genome
mkdir -p examples/hg19
mkdir -p universes/hg19

## download example universe and user set
curl http://cloud.databio.org.s3.amazonaws.com/vignettes/lola_vignette_data_150505.tgz | tar xvz

## move example universe and user set files to hg19 dir
mv lola_vignette_data/activeDHS_universe.bed universes/hg19/.
mv lola_vignette_data/setB_100.bed examples/hg19/.

## clean up
rm -rf lola_vignette_data

## download databases
curl http://cloud.databio.org.s3.amazonaws.com/regiondb/LOLACoreCaches_170206.tgz | tar xvz
curl http://cloud.databio.org.s3.amazonaws.com/regiondb/LOLAExtCaches_170206.tgz | tar xvz

## move databases to appropriate spots
mv scratch/ns5bc/resources/regions/LOLACore databases/Core
mv scratch/ns5bc/resources/regions/LOLAExt databases/Extended

## clean up
rm -rf scratch

## change ot local data dir
cd $LWLOCAL

## create placeholder dirs for cache and shinylog
mkdir -p cache
mkdir -p shinylog

## run the docker image
docker run -d \
  -p 80:80 \
  -e LWREF=$LWREF \
  -e LWLOCAL=$LWLOCAL \
  --volume ${LWLOCAL}:${LWLOCAL} \
  --volume ${LWREF}:${LWREF} \
  --volume ${LWLOCAL}/shinylog:/var/log/shiny-server \
  databio/lolaweb

## navigate to http://localhost/LOLAweb/apps/LOLAweb/

[fvalle1]

hi @vpnagraj thank you for your interest in this.

unfortunately I am still not able to run an instance of your code properly.

With this new instructions the web interface seems to correctly load example/universes/region databases, but it crashes as soon as I click "RunLola"

Is it possible that I am missing files in $LWREF/databases/Core/hg19/cistrome_cistrome/regions/ ? Comparing to your server they are missing after the setup process you proposed, is this expected?

[vpnagraj]

when testing on my end, im able to run the instructions above as-is (without the regions/ dir you mentioned) and process the example data successfully

one additional configuration change i would recommend is increasing the RAM allocated to the container. some of the cache files are quite large and processing is memory intensive. from your description it sounds possible that you are running out of memory. when i tried setting the memory limit to anything < 7GB RAM locally (using Docker Desktop) i was seeing the server crash with the following error in the log file at $LWLOCAL/shinylog/LOLAweb-shiny-{timestamp}.log

...
Calculating unit set overlaps...
Calculating universe set overlaps...
Killed

with the memory allocation in Docker Desktop set to 8GB i am seeing the example data processing succeed.

if you're running on a linux machine (without Docker Desktop) you may be able to adjust the memory allocation with a flag to the running container or by configuring the daemon settings.

please let us know if adjusting the memory allocation resolves the issue. if so, we'll go ahead and document the resource requirements accordingly.

[fvalle1]

My docker container has 16GB of RAM available..

I still receive the same error

[vpnagraj]

ok thanks for confirming that this is not a memory allocation issue.

re-reading your original post i noticed that your LWREF and LWLOCAL vars did not include a trailing slash:

I followed the instruction for a docker installation, then I set LWREF (/home/user/Developer/LOLAweb) and LWLOCAL (/home/user/Developer/LOLAweb/data) environment variables.

the example code included in my response mentioned the trailing slash, but i realized i didnt explicitly mention that was required in my comment.

have you added a trailing slash to your LWREF and LWLOCAL vars? im working on a fix to make the app processing of those env vars work with or without trailing slash ... but for now it is required.

[fvalle1]

Yes, my env variables had trailing slashes

Here my complete log, hope it helps LOLAweb-shiny-20220307-080118-44951.log

When trying mm10 examples LOLAweb-shiny-20220307-090550-42701.log

[fvalle1]

Hey @vpnagraj good news

I am now able to run the instructions in your comment; the problem on my side was related to docker not having all the right reading permission on $LWREF

Once I fixed it the computation started properly, but failed after Calculating Fisher score; here is the log

Calculating unit set overlaps... Calculating universe set overlaps... Calculating Fisher scores... Timing stopped at: 90.25 9.947 98.71 Warning: Error in aggregateOverGenomeBins: could not find function "aggregateOverGenomeBins" 119: system.time [/srv/shiny-server/LOLAweb/apps/LOLAweb/app.R#601] 118: eventReactiveHandler [/srv/shiny-server/LOLAweb/apps/LOLAweb/app.R#514] 74: rawdat_nocache 72: observeEventHandler [/srv/shiny-server/LOLAweb/apps/LOLAweb/app.R#387] 1: runApp

I was able to fix it updating the R packages and reinstalling GenomicDistributions using BiocManager

install.packages(c("ggplot2", "shiny", "DT", "shinyWidgets", "shinyjs", "sodium", "devtools", "shinyBS"))
devtools::install_github("databio/simpleCache")

if (!require("BiocManager", quietly = TRUE))
    install.packages("BiocManager")

BiocManager::install("GenomicDistributions")
BiocManager::install("GenomicRanges")

Now I am able to run your software properly!

Thank you for your assistance

[vpnagraj]

great! very glad to hear you were able to get everything running.

thanks again for raising this issue. we'll be improving our Docker documentation and updating the image to use the most current version of GenomicDistributions.

closing.