Proposal: create a user-facing benchmarking extension

Problem

Profiling of the UI performance issues is difficult due to differences in:

browsers and browser versions
sets of extensions users have installed, some of which are not even public
notebooks, cell outputs and other main area widgets used
complexity of workspaces

Some browser-specific performance regressions are caused by CSS rules which affect the number of elements marked for style recalculation; existence of a single such a rule in an extension will cause JupyterLab to slow down substantially unless browser implements clever optimisation strategies. Such issues by definition cannot be detected on CI.

Proposed Solution

I would like to propose a creation of user-facing extension enabling users and managers of JupyterLab deployments to benchmark the UI of JupyterLab locally, providing information on performance impact of installed extensions in a specific browser of choice for a set or pre-specified actions. The extension would ideally be forward compatible with upcoming Jupyter Notebook v7.

I would like to enable users/workspace administrators to:

run benchmarks for pre-specified UI actions such as (see https://github.com/jupyterlab/benchmarks/issues/84):
- opening context menu
- switching menus in the menu bar
- switching tabs
- opening notebooks
- opening completer
parameterise the benchmarks (e.g. using my_notebook.ipynb for tab switching)
generate results to share with developers (of extensions or of JupyterLab)
- automatic or semi-automatic uploading of reports is a non-goal as that would require some kind of privacy compliance checks, we don't really need that
- the result should optionally allow to include (allowing to deselect each of the options separately):
  - count of HTML nodes on the page, broken down by type
  - basic, anonymised workspace information (how many notebooks, how many cells, how many files were opened etc)
  - basic information about the browser/OS versions
run the benchmarks in any supported browser to be able to exactly capture the performance issue the users are facing
the benchmarks would cover:
- timing with specific stylesheets disabled to narrow down the plausible source of the problem
- timing specific style rules disabled
- JavaScript self-profiling for browsers which support it (currently only Chromium-based browsers) to understand if the problem comes from JS code, or any specific browser operation like layout, style or painting;
  - self-profiling would also require a server extension to enable self-profiling, see https://github.com/jupyter-server/jupyter_server/issues/1029

I don't know how to name it, but for now we could name the npm package @jupyterlab-benchmarks/ui-profiler and the PyPI jupyterlab-ui-profiler.

Some UI ideas for the UI profiler:

it would be accessible from the Help menu bar menu with an option to disable/re-enable the menu entries from settings/command palette
the results would be displayed in a main area widget; for simplicity that could be an HTML render of static output or a notebook

I already have a PoC for the CSS benchmarking part, and if we are happy to proceed with the idea of making it a user-facing extension, I would like to wrap it in an extension, integrate with CI (for the part which can be integrated) and open a PR.

Additional context

Another project VS Code, according to this wiki on GitHub:
- includes a self-profiler to record performance of extensions,
- allows users to easily report workspace metadata etc,
- for the renderer process it relies on the Chromium JS profiler.

jupyterlab / benchmarks