Container for Running Extasy Workflows on BW

[FranklinBetten]

Pros

1. container will avoid using BW_python and will be self contained with only specified libraries. It will use the libraries that we knwo work on "local" machine installs like on my laptop. This will then not change as modules on BW are updated - ideally would mean no redeveloping the work flow.
2. other potential benefit is we can maybe run multiple Extasy jobs at teh same time on BW without having to have a VM for each instance.

Cons

1. Perforamnce hit?
2. we have not used them much as a group so takes learning.

---

Singularity is supported well on BW and has extensive documentation https://singularity.lbl.gov/

-It uses the host machines ~/$USER/ folder by default so it is pretty simple to pipe files in and out of

• can build one working instance of this container "publish" it to the singulartyhub and can pull on any machine very easily.
• writable containers and interactive sessions - need to be able to set size of container larger than default bec will run out of space on installation of even just python2

Current status in Testing on Jetstream VM

[vivek-bala]

As I mentioned in the email, I am not really of anyone else using containers on BW right now, so the knowledge/experience on this would be limited. We do have multiple users who are using cybertools on BW without containers and see no deployment issues (except the few python related changes by BW).

Quick note that if you are looking into container-based solutions, I would recommend making sure that there is some type of replica controller support as well that allows you to deploy containers easily (and automatically re-deploy on errors/failures).