Live demo

[nsheff]

Would be cool to wrap up a docker container and deploy it in a little interactive demo.

Maybe load up some basic ATAC-seq sample data and put PEPATAC on there.

We could have a cron job that would remove all results once per hour or something.

[nsheff]

First step: build an all-inclusive docker container that you could use to run the demo locally.

start with the same way we're doing it for episb-hub: https://github.com/databio/episb-hub

Next step: making it so that PEPATAC can actually run (ideally linking to that as separate container).

[nsheff]

start with a flask dockerfile, like this:

https://github.com/databio/episb-hub/blob/master/Dockerfile

add docker to it, maybe something like this? https://github.com/docker-library/docker/blob/98ffef81ebfa7601a9ed2f0bf56d78f426bf253c/19.03-rc/Dockerfile

So now our container also can run docker (within the container)... we use the strategy desscribed here to mount the docker 'socket' from teh host into the container:

https://itnext.io/docker-in-docker-521958d34efd

using:

-v /var/run/docker.sock:/var/run/docker.sock

So now we run caravel in this container... point it to a default caravel.yaml with a few PEPs that point to a cloned pepatac repository... and set the default divvy config to use docker.

Now when someone runs caravel, the hit the caravel container, which shows them them projects and allows them to run them using the divvy docker template... this will create a new pepatac container for each sample, running on localhost

This leads to some issues we still need to resolve with the run command and how it communicates results to caravel, so we don't have to wait....

but in the meantime we can still solve all the docker demo stuff.

[nsheff]

@nmagee -- caravel is a flask app, like episb-hub. we'd like to enable a demo and run it in a container, like episb-hub is doing. But, we use it in a slightly different matter.. we start it with: caravel -c config.yaml on the command line, while episb-hub is started with flask run...

Do you know how we should combine these two ideas so that we can both offer caravel -c ... type running, but also easily run it in a container using the same setup that episb-hub uses?

[nsheff]

@michalstolarczyk can we use the same approach you developed for refgenie to run caravel in a container here?

[nsheff]

@michalstolarczyk now that we have the demo pretty well settled, can you put together some basic instructions for the container so we can pass it to Neal to set up a live version?

[stolarczyk]

@nmagee,

the Dockerfile in question can be found here: https://github.com/pepkit/caravel/blob/master/Dockerfile

we just need to build the image:

docker build -t caravel_demo .

and then run the container with caravel --demo, so:

docker run -p 5000:5000 caravel_demo caravel --demo

the port is a tunable parameter (-p or --port) of the caravel app, no volumes need to be mounted since the data used by the demo is distributed with the caravel package

[nmagee]

@michalstolarczyk a copy of the demo is running here: http://caravel.uvadcos.io/?token=QEDVSQ84Z7ZVBWT

I wonder if it would be much work, when using the --demo flag, if a fixed token could be set? That way remote relaunches of the container service in DCOS won't keep giving you different tokens each time?

http://caravel.uvadcos.io/?token=DEMO or something predictable?

[stolarczyk]

token-less mode is already implemented and can be accessed in two ways:

1. (preferred) use a YAML dotfile .caravel_token in the dir where caravel is run from

   token:
   - caravel_demo_token

2. run in debug mode with -d or --demo

[nsheff]

he's already running it in --demo mode. Look:

caravel --demo

Caravel is protected with a token.
Copy this link to your browser to authenticate:

 http://localhost:5000/?token=7CY745V8GFWHPUO

 * Serving Flask app "caravel.caravel" (lazy loading)
 * Environment: production
   WARNING: Do not use the development server in a production environment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)

[stolarczyk]

oh, I meant "-d or --dbg" here:

run in debug mode with -d or --demo

[stolarczyk]

So we can combine demo and debug modes to get demo data in the debug mode:

caravel --demo --dbg

or just use the .caravel_token file with:

caravel --demo

[nmagee]

@michalstolarczyk thanks. That did the trick.

http://caravel.uvadcos.io/?token=caravel_demo_token

[nsheff]

I clicked "shutdown", which shut down the server. We need to think through this a bit.

[nsheff]

michal the demo can't actually run anything; does the container lack sh or something?

[stolarczyk]

It might be related to the inability to expand envirnment variable ${HOME}

This is a line from looper run log/action page

Folder doesn’t exist for sample Sample ‘frog1’: ${HOME}/caravel_demo/results_pipeline/frog1

This issue does not occur when I run the container locally. Why would it behave differently there?