search

tskit-dev / msprime

Simulate genealogical trees and genomic sequence data using population genetic models
GNU General Public License v3.0
173 stars 86 forks source link

Make Binder links work for docs #1413

Closed jeromekelleher closed 3 years ago

jeromekelleher commented 3 years ago

Currently Binder links don't work. See JupyterBook docs on the subject here: https://jupyterbook.org/interactive/launchbuttons.html

jeromekelleher commented 3 years ago

Need to update the docker image

jeromekelleher commented 3 years ago

The Dockerfile seems to build, but it gets stuck on 

Launching server...
Launch attempt 1 failed, retrying...
Launch attempt 2 failed, retrying...

Maybe if we use a simpler Dockerfile it'll work a bit better.

jeromekelleher commented 3 years ago

The new docker file is much quicker to build and the server now starts really quickly. However, we now get a 404 in the Jupyter hub, at a url like https://hub.gke2.mybinder.org/user/tskit-dev-msprime-sklo4llj/tree/docs/ancestry.md

Anyone want to dig into this?

grahamgower commented 3 years ago

The "tree" is empty (and readonly). My guess is that because your docker image is installing the msprime wheel, the image doesn't include the git repository. Compare this with pointing mybinder.org at any repository without it's own Docker file (e.g https://mybinder.org/v2/gh/tskit-dev/tskit/main).

grahamgower commented 3 years ago

All the Binder links in the docs do is point mybinder.org at the msprime repository. Mybinder uses repo2docker to build the docker image it runs. There are several ways to configure a repository for repo2docker. For a Python package, the simplest thing is to have a pip requirements.txt file at the top-level, or if that's insufficiently flexible, include a conda environment.yml file (which has higher precedence). For maximum configurability, one can instead have a Dockerfile at the top-level, which has the highest precedence.

Presumably the Dockerfile could be fixed to work fine with mybinder, but I have no expertise to offer here. So given @agladstein is using the Dockerfile separately to mybinder, the simplest thing we could do is rename/move the Dockerfile and add a small environment.yml. Something like this:

name: name-is-ignored
dependencies:
  - python=3.9
  - pip
  - pip:
    # Pin specific pypi dependencies (not necessary, but probably desired).
    #- -r requirements/development.txt
    # Install from current directory.
    - .

Such a setup is working fine in https://github.com/popsim-consortium/demes-python and https://github.com/grahamgower/demesdraw.

grahamgower commented 3 years ago

Oh, and I just found this which explains how to get jupyter-book myst markdown files to open nicely in binder.

jeromekelleher commented 3 years ago

That sounds like a good plan, thanks @grahamgower. @agladstein, what will happen if we move the Docker image into directory? Can we update the config on Docker Hub to point to a custom location? I think it would be good to keep the default Docker image minimal, as you say in #1569 and we can make the Binder links work with more python specific stuff.

agladstein commented 3 years ago

I'm not sure about pointing Docker Hub to a directory, I would think that it should be doable. I'll try to take a look at this this weekend.

jeromekelleher commented 3 years ago

Any thoughts on what we should do here @agladstein?

agladstein commented 3 years ago

Sorry for being slow. I forgot to check last weekend.

Looks like it should work. Here is what I see in the build configuration on DockerHub

Screen Shot 2021-04-07 at 6 55 58 PM

Looks to me, like you'd just have to change the Dockerfile location to the directory. We can try a test in a branch if you like. But I think we'd have to test from tskit-dev/msprime and not a fork.

jeromekelleher commented 3 years ago

OK, great. How about I create a directory called containers and rename the current Dockerfile to containers/docker-minimal?

I'm not too worried if the Docker build is broken for a few days while we sort it out.

Looks like we need to update the config to point to main rather than master as well.

agladstein commented 3 years ago

that sounds good. Can you ping me here when the change is made on main? Then I'll try to get DockerHub to work with it.

jeromekelleher commented 3 years ago

I've moved the file @agladstein - also, we need to update it so that it's more minimal. Is there a minimal Python 3.7 (say) image we could use instead? (See #1569)

jeromekelleher commented 3 years ago

OK, I've added an environment.yml and the Binder server is starting up, but we're now just looking at a markdown file. Do you know how to get it to convert automatically to a jupyter notebook @grahamgower?

This is seeming like more work that it's worth, tbh

grahamgower commented 3 years ago

OK, I've added an environment.yml and the Binder server is starting up, but we're now just looking at a markdown file. Do you know how to get it to convert automatically to a jupyter notebook @grahamgower?

This is seeming like more work that it's worth, tbh

Something like #1635. But the notebooks based on markdown have artefacts that make them non-ideal. Not sure if there's a sane approach to hiding cells, etc. that looks good in jupyter-book and in the corresponding notebook.