search

jupyter-server / jupyter-resource-usage

Jupyter Notebook Extension for monitoring your own Resource Usage
BSD 2-Clause "Simplified" License
494 stars 102 forks source link

Jupyter Resource Usage Roadmap #58

Open jtpio opened 4 years ago

jtpio commented 4 years ago

Let's keep discussions on what to do next with nbresuse in this meta issue.

shreddd commented 4 years ago

May also need to reflect official clients in the roadmap (i.e. statusbar) that need to consume these metrics

Specifically - this endpoint will need to change: https://github.com/jupyterlab/jupyterlab/blob/4fe4dcfe5c9dc329bed2dcf2602f569ddef8a8a0/packages/statusbar/src/defaults/memoryUsage.tsx#L291

Noting it here since it is a cross repo change.

jtpio commented 4 years ago

Thanks @shreddd, added to the list.

kevin-bates commented 4 years ago

Given that per kernel metrics are in high demand, could we consider the possibility that this be baked directly into jupyter_server? Moreover, I'd love to see the kernel metrics just extend the existing kernel activity API, with either extending/api/kernels/{kernel-id} or via a filter parameter (e.g., /api/kernels/{kernel-id}?metrics) or a slightly new endpoint (/api/kernels/metrics/{kernel-id}). Of course, the advantage of the former is that no applications would need to change (thinking Jupyter Kernel/Enterprise Gateway) and/or add support for server extensions.

The server metrics could remain at /metrics or become /api/server/metrics (for possible other endpoints off /api/server) but would be separate from the kernels if that's possible.

We'll also want to plumb the gathering of metrics into MappingKernelManager - which has access to the kernel process (for local kernels) via the KernelManager instance. Eventually, either via a process abstraction layer or Kernel Providers, we'll have a specific method to call to gather metrics and let the abstraction layer/provider to the right thing, including metrics for remote kernels, containerized kernels, Spark driver kernels (w/ executors), etc.

It feels like multiple metrics implementations are popping up and it would be good to have a single destination/goal of where this should land.

cc: @ZSailer @lresende

jtpio commented 4 years ago

Given that per kernel metrics are in high demand, could we consider the possibility that this be baked directly into jupyter_server?

Yes absolutely :+1:

For now it was still unclear whether this should stay in its own project until it matures, be moved to the jupyterhub organization, or something else. But I agree that if the demand is increasing it would be better to have a more streamlined support, and having it backed into jupyter_server should help.

Also repo2docker recently added nbresuse by default so the memory usage can be displayed automatically in the classic notebook interface and JupyterLab status bar: https://github.com/jupyter/repo2docker/pull/904

Moreover, I'd love to see the kernel metrics just extend the existing kernel activity API, with either extending/api/kernels/{kernel-id} or via a filter parameter (e.g., /api/kernels/{kernel-id}?metrics) or a slightly new endpoint (/api/kernels/metrics/{kernel-id}). Of course, the advantage of the former is that no applications would need to change (thinking Jupyter Kernel/Enterprise Gateway) and/or add support for server extensions

It sounds like extending the existing kernel endpoints would be convenient.

/metrics will have to be removed at some point to not shadow the Prometheus endpoint.

It feels like multiple metrics implementations are popping up and it would be good to have a single destination/goal of where this should land.

Should we list these other metrics implementations (for example in this issue) so we can keep an eye of them?

+1 for having a single source of truth for gathering kernel and server metrics.

kevin-bates commented 4 years ago

Awesome, it sounds like we're on the same page here!

Should we list these other metrics implementations (for example in this issue) so we can keep an eye of them?

Actually, I may be conflating the discussion from the lab issue and your work in jupyterlab-system-monitor with nbresuse a bit. But there was this thread of discussion in jupyter_client - which derived from another in notebook. Not to mention this Kernel Gateway question - but that's nbreuse-based as well.

So, although there may not be lots of implementations, there is definitely lots of interest and this should find its way into the server at some point.

mlucool commented 4 years ago

+1 for baking this into jupyter_server. Feels like a great feature to have out of the box, especially if it includes per kernel!

jtpio commented 4 years ago

Decide where the repo should live: keep under yuvipanda or more to another GitHub organization (or move to Jupyter Server)? #24

A candidate for now could be the recent jupyter-server organization, as a new home for this repo?