Add `placement` as a regular config option and not only for user_options

[isuftin]

As with other options we can choose for configuring the swarm deployment, I feel as though placement should be available as a configuration option not only to be set through user_options.

An example of what I mean...

As a configuration level option, it currently does not work: jupyterhub_config.py

[...]
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
c.SwarmSpawner.placement = ["node.role == worker"]
[...]

Launch logs:

jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | Adding user testuser
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.479 JupyterHub app:724] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.545 JupyterHub app:892] Not using whitelist. Any authenticated user will be allowed.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.560 JupyterHub configurable:168] Config option `placement` not recognized by `SwarmSpawner`.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.636 JupyterHub swarmspawner:229] Docker service 'jupyter-90613d4f8204b906ead85556f075fd71-1' is gone
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.636 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.640 JupyterHub configurable:168] Config option `placement` not recognized by `SwarmSpawner`.
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.644 JupyterHub swarmspawner:229] Docker service 'jupyter-2f4c77557cd6c786d40e488583d77e79-1' is gone
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [W 2017-08-11 15:38:18.645 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.652 JupyterHub app:1453] Hub API listening on http://0.0.0.0:8081/hub/
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.655 JupyterHub app:1176] Starting proxy @ http://0.0.0.0:443/
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | 15:38:18.859 - info: [ConfigProxy] Proxying https://0.0.0.0:443 to http://127.0.0.1:8081
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | 15:38:18.864 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes
jupyterhub_jupyterhub.1.m9kw855bogr9@manager    | [I 2017-08-11 15:38:18.964 JupyterHub app:1485] JupyterHub is now running at http://127.0.0.1:443/

But it does work if I add it under user_options and set to use user_options to true jupyterhub_config.py

[...]
c.JupyterHub.spawner_class = 'cassinyspawner.SwarmSpawner'
# c.SwarmSpawner.placement = ["node.role == worker"]
c.SwarmSpawner.use_user_options = True
user_options = {
    'placement' : ["node.role == worker"]
}
[...]

Launch logs:

jupyterhub_jupyterhub.1.879snisqard1@manager    | Adding user testuser
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.836 JupyterHub app:724] Loading cookie_secret from /srv/jupyterhub/jupyterhub_cookie_secret
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.893 JupyterHub app:892] Not using whitelist. Any authenticated user will be allowed.
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.973 JupyterHub swarmspawner:229] Docker service 'jupyter-90613d4f8204b906ead85556f075fd71-1' is gone
jupyterhub_jupyterhub.1.879snisqard1@manager    | [W 2017-08-11 15:47:19.973 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.879snisqard1@manager    | [W 2017-08-11 15:47:19.975 JupyterHub app:1097] isuftin not running.
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.983 JupyterHub swarmspawner:229] Docker service 'jupyter-2f4c77557cd6c786d40e488583d77e79-1' is gone
jupyterhub_jupyterhub.1.879snisqard1@manager    | [W 2017-08-11 15:47:19.984 JupyterHub swarmspawner:194] Docker service not found
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.996 JupyterHub app:1453] Hub API listening on http://0.0.0.0:8081/hub/
jupyterhub_jupyterhub.1.879snisqard1@manager    | [I 2017-08-11 15:47:19.999 JupyterHub app:1176] Starting proxy @ http://0.0.0.0:443/
jupyterhub_jupyterhub.1.879snisqard1@manager    | 15:47:20.208 - info: [ConfigProxy] Proxying https://0.0.0.0:443 to http://127.0.0.1:8081
jupyterhub_jupyterhub.1.879snisqard1@manager    | 15:47:20.213 - info: [ConfigProxy] Proxy API at http://127.0.0.1:444/api/routes

[barrachri]

Hi @isuftin , how can you say that it works? Because you miss this line in the logs Config option "placement" not recognized by "SwarmSpawner"?

[isuftin]

@barrachri The line you refer to is when the configuration uses the placement as a top-level configuration option instead of adding it into the user_options dictionary. When used as a top-level configuration option is when DockerSpawner complains. My feature request is to have it added as a top-level configuration option.

Unless you're saying that even when used in the user_options dictionary, it doesn't work?

Sorry, your statement was slightly difficult to read.

[barrachri]

Hi @isuftin sorry, my message was a bit unclear.

But you got my point:

Unless you're saying that even when used in the user_options dictionary, it doesn't work?

DockerSpawner complains because placement is not recognised as valid option, in your second example the option is not just used at all, it's just a dict inside the config.

I think it's easy to implement this feature, would like to submit a PR?

[isuftin]

@barrachri I can try to submit a PR once I have a little time to do so. I hope to do so this week.

[barrachri]

@isuftin thanks :)

[isuftin]

The PR is in and I've tested it as working. With the following set in jupyterhub_config.py:

c.SwarmSpawner.placement = ["node.role == worker"]

I can inspect the launched notebook process via docker inspect <service id> and I can see:

       "Spec": {
            "Name": "jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1",
            "Labels": {},
            "TaskTemplate": {
                "ContainerSpec": {
                [...]
                "Placement": {
                    "Constraints": [
                        "node.role == worker"
                    ]
                }
                [...]

$ docker node inspect worker1 | grep worker
            "Role": "worker",
            "Hostname": "worker1",

$ docker service ls
ID                  NAME                                         MODE                REPLICAS            IMAGE                                           PORTS
5ub2qq4lt1ho        registry_registry                            replicated          1/1                 registry:2.6.2                                  *:5000->5000/tcp
kvm3thjufitn        jupyterhub_jupyterhub                        replicated          1/1                 localhost:5000/dlcc/jupyterhub:latest           *:443->443/tcp
u4waxtqq1t3r        jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1   replicated          1/1                 localhost:5000/dlcc/jupyter-singleuser:latest

$ docker service ps u4
ID                  NAME                                           IMAGE                                           NODE                DESIRED STATE       CURRENT STATE           ERROR               PORTS
wysn1oijgmdp        jupyter-5d9c68c6c50ed3d02a2fcf54f63993b6-1.1   localhost:5000/dlcc/jupyter-singleuser:latest   worker1             Running             Running 9 minutes ago

Working as expected!

[barrachri]

Thanks!

merged.