holoviz-topics / examples

Visualization-focused examples of using HoloViz for specific topics
https://examples.holoviz.org
Creative Commons Attribution 4.0 International
80 stars 24 forks source link

Add Lumen Monitor example #335

Open jbednar opened 8 months ago

jbednar commented 8 months ago

The examples collected in this repo are currently deployed on an Anaconda Enterprise cluster that manages deployments, and we have an internal application based on Lumen that can monitor those deployments and the nodes they are running on to see usage patterns, memory leaks, memory requirements, etc. It would be great to add that Lumen app here as an example of using Lumen to do such monitoring, since most of what it does is independent of AE5 and valid as an illustration of monitoring separate deployments in general.

image
maximlt commented 8 months ago

Yes that's good idea.

Although there's a limitation in that this project needs to declare some secrets as environment variables (e.g. AE5_PASSWORD) but we have no way yet to set Github environment variables and pass them to anaconda-project, either for testing/building the project or deploying to AE5. For sure we could inject environment variables set in Github into an anaconda-project file (e.g. relying on a naming scheme to find them like EXAMPLES__<PROJNAME>__<VARNAME>), this needs some infrastructure development.

I also wonder how we will demonstrate this example from a notebook. Maybe using Lumen's Python API to incrementally build the monitor dashboard, to finally spit out the YAML spec.

If there's a goal of adding any kind of Lumen example to the gallery, there are certainly easier examples. But I also like the idea of having the Monitor as an example, that can be tested, built and deployed from this Github repo.

jbednar commented 8 months ago

I'm not sure how to handle the secrets, but it's worth having an answer to that, since projects like this do often need secrets. I don't think we have to demonstrate the example from a notebook, as we can simply add a notebook that documents the app and explains it. It doesn't have to be runnable within the notebook itself, as there will be a link to the running app.