search

krishnan-r / sparkmonitor

Monitor Apache Spark from Jupyter Notebook
https://krishnan-r.github.io/sparkmonitor/
Apache License 2.0
172 stars 55 forks source link

Adding compatibility with JupyterLab #34

Closed itsjafer closed 4 years ago

itsjafer commented 4 years ago

This PR updates the previous sparkmonitor code to be compatible with JupyterLab.

A summary of changes made:

  1. The backend scalalistener, kernel, and server extensions were relatively untouched except for general linting fixes. On top of this, the scalalistener was recompiled and pushed to the repo so that it is no longer necessary to recompile for basic front-end changes.
  2. The front-end was linted for formatting and syntax rules. On top of this, much of the front-end kernel communication was rewritten. Furthermore, the way jupyter events were tracked and listened to has been updated to work with JupyterLab.
  3. I've added pre-commit, tox, and a few other tools that I've become accustomed to while working at Yelp to this repo in order to make the development process smoother.

Here are what changes will be present in the next PR:

  1. Currently, clicking on tasks will not open a dialog with the default Spark UI. I don't suspect this change will take long, but since it can be done independent of this PR, I will publish it in the next PR. You may notice the code for this is present but commented out at present.
  2. In the original notebook extension, there is a button for toggling all SparkMonitors. I will add this in the next PR as well.
  3. An updated README with instructions on development, quick-start. At the moment, the easiest way to test the extension is by entering the extension folder and running make build followed by make develop. I received permission from @krishnan-r to maintain this repo for the jupyterlab version of this extension so I intend to remove notebook-related information and link the jupyter notebook version of this extension in the updated README.
  4. Updated jsdocs (potentially)