Logging out, and releasing resources

[nbarlowATI]

One thing to think about is how to handle users logging out, and when/how to stop/deallocate compute instances. Jupyterlab doesn't have a logout button, but could perhaps have an extension to do this. Then, what should happen when a user logs out? They may want to login again soon and carry on working on the same compute instance, but if we always leave the VMs running, this would be expensive.

[nbarlowATI]

Another minor thing that @pierocor has observed is that after logging out (closing Jupyterlab tab), when logging back in again, the user goes straight back to the running CI, skipping the (new) workspace-select screen, which is not the desired behaviour. Maybe cookie-related?

[tam203]

Hi @nbarlowATI , we discussed this a little bit to re-iterate.

The behaviour you describe is the default behaviour on Jupyter Hub / Pangeo systems:

• Log out is completely separate from stopping your compute. You can do either independently.
• By default Jupyter Hub will redirect you to your started/spawned instance if it's up and running. "By default, in a JupyterHub deployment each user has exactly one server." - so it makes sense to redirect to it if it's running.
• If we intend to have multiple servers (Az ML compute instances (in one or more Work Space)) we should use named-servers
• Users should be able to manage their compute instance at <jupyterhub-url>/hub/home - It should offer start/stop (and spawn new server if named servers are enabled).

So. I think what we have is inline with the expected behaviou of Jupyter hub. Things we should do however to improve the situation:

• [x] Use named servers for spawning either different compute instances (compute size S/M/L) or in a different workspace (project)
• [ ] ensure that stop and start works as expected at <jupyterhub-url>/hub/home
• [ ] Investigate if there is information we can get from AML that would alow us to implement a 'culler'. A 'culler' kills user servers if they have been 'unused' for a set amount of time. This might not be possible at the moment. (Would need the AML API to offer a 'last accessed' or similar time).

[tam203]

@pierocor - in middle of checking stop stat works correctly.

If yes then close.

Add a comment about 'clean up servers'

[pierocor]

Stopping a running named server form/hub/homesends signal SIGTERM to JH:

[D 2020-11-09 13:33:55.070 JupyterHub user:739] Stopping piero coronica:A-small
[I 2020-11-09 13:33:55.160 JupyterHub aml_spawner:289] Stopping the compute instance.
[I 2020-11-09 13:34:00.736 JupyterHub aml_spawner:323] Stopping the redirect server route: ('0.0.0.0', 9001).
WARNING:asyncio:Executing <Task pending name='Task-74' coro=<BaseHandler.stop_single_user.<locals>.stop() running at /home/centos/.conda/envs/azml/lib/python3.8/site-packages/jupyterhub/handlers/base.py:1052> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x7fd83aaaecd0>()] created at /home/centos/.conda/envs/azml/lib/python3.8/asyncio/base_events.py:422> cb=[chain_future.<locals>.copy() at /home/centos/.conda/envs/azml/lib/python3.8/site-packages/tornado/concurrent.py:154, with_timeout.<locals>.<lambda>() at /home/centos/.conda/envs/azml/lib/python3.8/site-packages/tornado/gen.py:614] created at /home/centos/.conda/envs/azml/lib/python3.8/site-packages/jupyterhub/handlers/base.py:1081> took 5.776 seconds
[C 2020-11-09 13:34:00.850 JupyterHub app:2684] Received signal SIGTERM, initiating shutdown...
/home/centos/.conda/envs/azml/lib/python3.8/site-packages/jupyterhub/app.py:2686: DeprecationWarning: Task.all_tasks() is deprecated, use asyncio.all_tasks() instead
  t for t in asyncio.Task.all_tasks() if t is not asyncio.Task.current_task()
/home/centos/.conda/envs/azml/lib/python3.8/site-packages/jupyterhub/app.py:2686: DeprecationWarning: Task.current_task() is deprecated, use asyncio.current_task() instead
  t for t in asyncio.Task.all_tasks() if t is not asyncio.Task.current_task()

[tam203]

When using named servers (as we are doing). You can 'stop' or 'delete' a compute instance. Currently delete is the same as stop but it would be interesting if we could have delete delete the instance and stop just stop it.

Have rased this in gitter: https://gitter.im/jupyterhub/jupyterhub?at=5fa948b2d5a5a635f2ae36d3