Dockerize loom

[slinnarsson]

Instead of reading the thesis I’m opposing tomorrow I took the opportunity to procrastinate by programming. I made an initial dockerized version of Loom, i.e. one that runs in a container. You can try it at http://104.196.41.157. I had to make many changes to the structure of the project and how it’s built, so I made a git branch “dockerized”.

Many remaining issues:

• Write documentation, including how to get it up and running on a new development computer
• Figure out how to run locally for testing and debugging (without need to download .loom files each time). E.g. by building a cache folder into the container.
• Figure out how to update the running (deployed) version correctly.
• Why does the sparkline plot not work?
• Attach a block volume and place the cache there, so it's shared by all container instances
• Figure out how to update the cache (block volume) when new datasets are generated
• Dockerize additional processes: the loom creation pipeline, the C# analysis pipeline, ...
• Figure out how to use a secret to store the credentials (they are now in the repo, which is bad).
• Merge back into "master" branch

[JobLeonard]

I'm trying to get Docker to work on my laptop so we can merge. I think I have everything ready until the ./build step, but at that point it fails:

job@job-ThinkPad-T440s-Ubuntu ~/D/L/Loom> ./build
Building React in 'production' mode. Run "./build debug" to build in 'debug' mode
Building static assets.
Building docker containers.
Sending build context to Docker daemon 103.3 MB
Step 1 : FROM gcr.io/linnarsson-lab/loom-server-base:v1
Pulling repository gcr.io/linnarsson-lab/loom-server-base
Error: Status 403 trying to pull repository linnarsson-lab/loom-server-base: "Unable to access the repository: linnarsson-lab/loom-server-base; please verify that it exists and you have permission to access it (no valid credential was supplied)."

I probably overlooked some part of some installation documentation telling me to run a command to set those permissions, looking into it myself, but if you happen to know what I'm missing please tell

[slinnarsson]

Did you do “gcloud auth login” or “gcloud init”?

https://cloud.google.com/sdk/docs/authorizing

You will also need to do this:

gcloud config set project linnarsson-lab gcloud config set compute/zone us-central1-a gcloud config set container/cluster container-cluster-3 gcloud container clusters get-credentials container-cluster-3

...which may or may not already be done by “gcloud init”.

/Sten

Sten Linnarsson, PhD Professor of Molecular Systems Biology Karolinska Institutet Unit of Molecular Neurobiology Department of Medical Biochemistry and Biophysics Scheeles väg 1, 171 77 Stockholm, Sweden +46 8 52 48 75 77 (office) +46 70 399 32 06 (mobile)

On 28 Apr 2016, at 14:32, Job van der Zwan notifications@github.com<mailto:notifications@github.com> wrote:

I'm trying to get Docker to work on my laptop so we can merge. I think I have everything ready until the ./build step, but at that point it fails:

job@job-ThinkPad-T440s-Ubuntu ~/D/L/Loom> ./build Building React in 'production' mode. Run "./build debug" to build in 'debug' mode Building static assets. Building docker containers. Sending build context to Docker daemon 103.3 MB Step 1 : FROM gcr.io/linnarsson-lab/loom-server-base:v1http://gcr.io/linnarsson-lab/loom-server-base:v1 Pulling repository gcr.io/linnarsson-lab/loom-server-basehttp://gcr.io/linnarsson-lab/loom-server-base Error: Status 403 trying to pull repository linnarsson-lab/loom-server-base: "Unable to access the repository: linnarsson-lab/loom-server-base; please verify that it exists and you have permission to access it (no valid credential was supplied)."

I probably overlooked some part of some installation documentation telling me to run a command to set those permissions, looking into it myself, but if you happen to know what I'm missing please tell

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHubhttps://github.com/linnarsson-lab/Loom/issues/14#issuecomment-215409194

[JobLeonard]

I have run all of these commands, and again now to double-check that I didn't miss something.

I'll dig through the gcloud and docker documentation, maybe it's an easy-to-miss point of difference between a Linux and Mac installation

[JobLeonard]

I had to run gcloud docker once to log in.

I don't know why, actually, I just saw gcloud docker push gcr.io/linnarsson-lab/loom-server:v$v in the build script and typed gcloud docker to see what feedback I would get... (expecting some login prompt) and it magically fixed things.

Am I supposed to log-in and mount seven times, by the way? See full transcript of my shell here

[JobLeonard]

Either way, it seems to work so we can merge if you want!

[JobLeonard]

Or no, let's wait until I fixed the forms.

[slinnarsson]

After a lot of frustration working with containers, I've made a simple VM version, controlled by supervisord. It seems much easier to work with and you can ssh in and see what's going on. It's at http://104.197.181.233 for now until we decide if that's the way to go.