jupyter-server / jupyter-resource-usage

Jupyter Notebook Extension for monitoring your own Resource Usage
BSD 2-Clause "Simplified" License
495 stars 102 forks source link

Add a new /api/nbresuse/v1 endpoint #52

Open jtpio opened 4 years ago

jtpio commented 4 years ago

https://github.com/yuvipanda/nbresuse/pull/45 restored the /metrics endpoint, so that JupyterLab (and other frontends) can display the metrics:

image

The reason to move away from /metrics is to not shadow the default Prometheus endpoint (see https://github.com/yuvipanda/nbresuse/pull/22 for more infos).

However this is a breaking change. We should carefully plan the rollout to avoid confusing users with an older version of JupyterLab. For the classic notebook this is not really an issue as the notebook extension is bundled with the package on PyPI.

cc @Gsbreddy who has already started working on this in side PRs.

jtpio commented 4 years ago

For now the code in the API handler:

https://github.com/yuvipanda/nbresuse/blob/ba1f4e794082fbac6865cb8c36e38b03155423fb/nbresuse/api.py#L21-L60

Duplicates some of the logic of the Prometheus handler, which relies on a PSUtilMetricsLoader:

https://github.com/yuvipanda/nbresuse/blob/ba1f4e794082fbac6865cb8c36e38b03155423fb/nbresuse/metrics.py#L9

It would be nice if both could share the same approach when it comes to retrieving metrics from psutil.

jtpio commented 4 years ago

It would be interesting to have a look at the glances API: https://github.com/nicolargo/glances/wiki/The-Glances-RESTFULL-JSON-API

Glances relies on psutil to provide aggregated metrics from the host machine.

There might be some ideas that could be applied to nbresuse if we start expanding the API surface.

yuvipanda commented 4 years ago

https://github.com/yuvipanda/nbresuse/pull/68 provides /api/metrics/v1, without removing /metrics. We can probably tweak it more in a /api/metrics/v2

Gsbreddy commented 4 years ago

We can add the per kernel metrics https://github.com/yuvipanda/nbresuse/pull/41 since this has been a long ask for lots of people. I myself am using this in my org but was overriding /metrics to enable this capability. Now that the soft deprecate went it, I think we can integrate https://github.com/yuvipanda/nbresuse/pull/41 onto /api/metrics/v2. Also CPU per kernel as well is a big addition. What do you guys think? @yuvipanda @jtpio