Issues that need to be fixed to run Docker version successfully

[djinnome]

Thank you for providing the Docker instance.

I was able to run docker and start the jupyter notebook on my machine and with a bit of effort I was able to build the model and solve it using quadminos. I only tested the 0.0.7 implementation that was in the Manuscript subdirectory, and not the 0.0.8 version that is the current version. The instructions could be a bit more clear, so I will detail what needs to be fixed in order to get it to work: 

1. run_jupyter.sh was in /home/meuser/docker, not /home/meuser as specified in the documentation 
2. I needed to run pip3 install --user . within the cobrame-0.0.7 directory in order to be able to import cobrame into the ecolime solve_demo.ipynb jupyter notebook 
3. Both the build_me_model.py script and the build_me_model.ipynb notebook had a bug in the call to return_gr_dependent_dna_demand(): 

Traceback (most recent call last):
  File "build_me_model.py", line 821, in <module>
    me = return_me_model()
  File "build_me_model.py", line 319, in return_me_model
    me.global_info['GC_fraction'])
TypeError: return_gr_dependent_dna_demand() missing 1 required positional argument: 'gc_fraction'

4. As a result, of this bug, the pickled model was never generated for solve_demo.ipynb to run. 

This was solved by adding me as the first parameter and me.global_info['GC_fraction'] as the second parameter to return_gr_dependent_dna_demand()

5. The last cell of solve_demo.ipynb notebook was unable to import escher because escher installation was not included in the Dockerfile. This was solved with pip install --user escher. 

6. The good news is that quadminos worked without a hitch, and I was able to view the results in Escher after fixing the bugs mentioned above.

[djinnome]

Also, you may want to consider using Binder instead of/in addition to docker, as it is much less of a technical bar for users (and reviewers) to install and get working. I actually learned about Binder from a tweet by @zakandrewking

[coltonlloyd]

Hi Jeremy,

Thank you for the thoughtful feedback!

I have thought about using Binder as a way to more easily deploy COBRAme. I've seen in the past that for larger/more computationally intensive projects Binder was pretty slow to initiate/run. This was a while ago, so maybe it has improved. I'll definitely look into it again. At a minimum we could use Binder to deploy a smaller educational version of the E. coli ME-model.

As for the docker errors, I think 2-4 may result from different versions of ECOLIme and COBRAme being used (ie. version 0.0.7 of COBRAme being build with version 0.0.8 of ECOLIme). Though this is not clearly stated in the documentation, the two repositories require that their versions are the same. I'll update the READMEs to make this clear.

I will add the installation of escher into the Dockerfile, along with other packages that I neglected to include. I'll also remove the run_jupyter.sh file and just replace it with an instruction in the README since its only contains a one line command. Thank you for catching these two errors.

Hope this helps. Please let me know if you have any other questions or comments.

[djinnome]

Sure, let me know when you are done, and I'll test it out. Perhaps we can avoid another round of reviews ;-)

[coltonlloyd]

Hi Jeremy,

Thank you for working with me on this! I just pushed a new commit that tried to make the Docker installation a bit more intuitive. The installed Docker image will already contain a model that was created by running the build_me_model.py script during the Docker building process. I thinks it's possible that last time an older version of build_me_model.py was ran using the newer version of ECOLIme that was installed in the Docker image, which was the source of some of the errors.

I also tried using Binder to launch a notebook from the Docker image and the build timed out before it could launch. For the time being I think the COBRAme + its dependencies are unfortunately too burdensome to run on the Binder/JupyterHub servers. I'll keep thinking of ways I could get this to work.

Thanks again for all your help.

[djinnome]

Hi Colton.

Thanks for making it possible to get up and running quickly on such a complex model!

So I pulled the latest cobrame, ran docker, and noticed a few things in the Docker environment that were still inconsistent with the documentation in the cobrame/docker/README.rst.

In the command line, run docker run -p 8888:8888 --rm -i -v $(pwd):/mount_point/ -t sbrg/cobrame:master bash. This will initiate a Docker container (virtual machine) into the /home/meuser directory and mount the contents of the directory where the command was ran into the docker container at /mount_point/.

However, when I ran docker run -p 8888:8888 --rm -i -v $(pwd):/mount_point/ -t sbrg/cobrame:master bash

I ended up in the /source directory, not the /home/meuser directory

Also, the documentation says:

The Docker image initializes in the /home/meuser directory which contains a json and pickle of iJL1678b in the /home/meuser/me_models directory which were created from running build_me_model.py while building the Docker image.

However, when I changed directory to /home/meuser the directory was empty except for the run_jupyter.sh bash file. I did not see a json or pickle file of iJL1678b.

I therefore went back to the /source directory, which does contain a copy of ecolime and cobrame (as well as solveme, cobrapy, qminos etc) and successfully started up jupyter notebook --ip=0.0.0.0 --port=8888.

From there, I was able to navigate to ecolime/ecolime/build_me_model.ipynb within jupyter notebook and successfully executed all the cells in the notebook.

I was also able to navigate to ecolime/ecolime/solve_demo.ipynb within jupyter notebook and successfully executed all the cells in that notebook except for the call to show_escher_map(me).display_in_notebook(), which resulted in the following error message:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-5-fed5a1b8c115> in <module>()
      1 # Display metabolic flux on escher map
----> 2 show_escher_map(me).display_in_notebook()

<ipython-input-3-f8b26235e77a> in show_escher_map(me, solution)
     14 
     15 def show_escher_map(me, solution=None):
---> 16     import escher
     17     view = escher.Builder("iJO1366.Central metabolism")
     18     view.reaction_data = me.get_metabolic_flux(solution=solution)

ModuleNotFoundError: No module named 'escher'

[djinnome]

Ah, nevermind. The problem was I didn't run

docker pull sbrg/cobrame:master

Just running docker pull sbrg/cobrame results in an error:

Using default tag: latest
Error response from daemon: manifest for sbrg/cobrame:latest not found

[djinnome]

OK, I have confirmed that after running jupyter notebook --ip=0.0.0.0 --port=8888 in /home/meuser and then running solve_demo.ipynb from within the jupyter notebook, I was able to successfully execute all the cells within the notebook, including show_escher_map(me).display_in_notebook().

If users or reviewers have already downloaded an older docker image, it would be wise to remind them to pull the latest version of the docker image with

docker pull sbrg/cobrame:master

[coltonlloyd]

That's a very good point. I'll add docker pull sbrg/cobrame:master to the installation steps since it's such an easy mistake to make.

Thanks again for taking the time to try this out!

[djinnome]

Sure, no problem.

Btw, I was looking at the instructions for using Docker with Binder, and it seems that there are some specific things you would need to modify about the current Dockerfile in order to get it to successfully run with Binder. For example, setting the specific tag of the Docker Images in the FROM statement, and issuing the default command with CMD statement to ensure that jupyter notebook is executed on startup.

Perhaps you already did these things when trying to run Binder, but if not, you might try giving it another shot.

[coltonlloyd]

I tried creating a new Dockerfile that only contained a FROM statement (w/ tag included) to pull the image from Dockerhub and the CMD statement to launch the Jupyter notebook. I think the image itself was just too large to download before timing out. I'll try some different things and see if I can find something that works.

[djinnome]

Hi Colton,

Congratulations on the publication! Now that the docker file seems to be working as advertised, perhaps it is unnecessary to get it working in Binder, but I am curious as to the status. Was it something you were ever able to get working, and if not, what was the main hurdle?