Zsailer commented 3 years ago

Hello everyone,

Welcome to the Jupyter Server Team!

You can find us in our Gitter chat room or visit us at our weekly meetings!

We meet on Thursdays at 8:00am, Pacific Standard Time on Jupyter's Zoom Channel. You can add yourself to the weekly agenda here. Everyone is welcome!

Let's avoid using this thread for discussion. If you'd like to discuss something in the minutes, open a separate issue and reference this thread.

Meeting Notes

Zsailer commented 3 years ago

January 7th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Steven Silvester	Apple	@blink1073
Afshin T. Darian	Two Sigma	@afshin

Agenda + Minutes

The pytest-jupyter repo - archive or clarify purpose?
- https://github.com/jupyter-server/pytest-jupyter/pull/10
- Archived the Github repo
  - Add archived message to README in repo.
- What do we do about the PyPI package?
  - Looks like we can delete from PyPI
  - Can we archive or deprecate?
New year, new list of meeting notes
- https://github.com/jupyter-server/team-compass/issues/4
- Let's leave the 2020 Notes issue pinned until the end of the month.
Name a core Jupyter Server Team
- https://github.com/jupyter-server/team-compass/issues/5
pytest-tornasync conflicts with pytest-tornado
- Alex tried to use Jupyter server's pytest plugin with nbdime, nbdime uses pytest-tornado, which just caused test to fail: https://github.com/jupyter/nbdime/pull/551#issuecomment-756018368
  
  def jp_http_port(http_server_port): E fixture 'http_server_port' not found
The dreaded open_browser trait
- https://github.com/jupyter-server/jupyter_server/pull/375
Released Jupyter Server 1.2.0
Add a repo for supplying MathJax resources (?)
- What should the name of this repo be?
  - jupyter_server_mathjax

Zsailer commented 3 years ago

January 14th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

Vidar:
- https://github.com/jupyter-server/jupyter_server_mathjax
- branch with traits. Is the mixin mattern the best way to add these trais config object? https://github.com/jupyter-server/jupyter_server_mathjax/blob/traits-mixin/jupyter_server_mathjax/traits.py
Kevin:
- Building Enterprise Gateway on server.
  - Finding a couple async-related issues in the session handler - will submit a PR today.
- Authoring Kernel Environment Provisioning proposal for jupyter_client

Zsailer commented 3 years ago

January 21st, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Pierre-Olivier Simonard	ioapps.io / Quansight	@pierrotsmnrd
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf
Gonzalo Peña-Castellanos	Quansight	@goanpeca
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

Port jupyter/notebook#5941—enable support for answer_yes flag
Pull Request #391 to add session management into Jupyter Server ( [name=Pierre] )
- Fantastic demo from Pierre; thanks to @echarles for opening the PR!
- There was some concern about the torndsession dependency. It hasn't been maintained much in the last few years.
- There was also some concern around whether sessions for multi-user experience pose security risks. Perhaps this should only be used with a fully trusted group of users.
Kernel Environment Provisioning Proposal; thank you, @kevin-bates!
- https://github.com/jupyter/jupyter_client/issues/608
- If people have time this next week, please take a look at this proposal.
Enable notebook ContentsManager subclasses in jupyter_server
- https://github.com/jupyter-server/jupyter_server/pull/392
- And the mirror of this in 'notebook' package:
  - https://github.com/jupyter/notebook/pull/5957
Re-enable default_url in ExtensionApps
- https://github.com/jupyter-server/jupyter_server/pull/393

Zsailer commented 3 years ago

January 28th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Afshin T. Darian	Two Sigma	@afshin
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf

Agenda + Minutes

Failing tests on Windows, Python 3.6
- Noticed in this PR: https://github.com/jupyter-server/jupyter_server/pull/392
- The underlying issue still hasn't been resolved.
  - Issue appears to be a race condition exposed by slower logging.
  - Updates will be posted when more is known.
ExtensionApp not compatible with extending ServerApp?
- Issue: https://github.com/jupyter-server/jupyter_server/issues/395
- Why doesn't ExtensionApp subclass ServerApp?
  - Jupyter Server is meant to be a headless server.
  - All extensions are equal in the eyes of Jupyter Server.
  - Today versus previously:
    - Previously, extensions (e.g. LabApp from JupyterLab) extended the server two different ways based on how it were launched:
      1. The extension module could include an extension application class (e.g. LabApp); it subclassed the server application. If the extension was launched through its entrypoint, this extension application class was used.
        
        The result is something like, LabApp(ServerApp).
      2. Alteratively, if the server application was launched and the extension was enabled, the extension application was not used; rather, just the the handler appending and config loaded.
        
        The result is something like ServerApp + "separate logic to appending handlers and load Lab specific config".
    - Today, extensions offers a single, consistent way to extend the server:
      1. Initialize the ServerApp, then initialize all ExtensionApps. Configure the server to decide which extension is the default.
        
        The result is always, ServerApp + ExtensionApp(s)
        
        ServerApp has an extension_manager; ExtensionApps have a serverapp property.
        
        ExtensionApps do have a launch_instance method. This is simply a shortcut to automatically configure and initialize a ServerApp. This allows ExtensionApps to behave like a server application, but not drastically change how they extend the server.
  - In the previous approach, one extension was elevated above the rest—the one that subclasses the server application.
  - Today, all extensions are handled the same.
    - This makes it cleaner to run a single Jupyter Server with e.g. nbclassic, jupyterlab, voila, etc. side-by-side.
    - ExtensionApps are not, themselves, server applications.
      - Their traits and methods are self encapsulated in their own class
      - They should be able to affect the core server application and append handlers.
- What's the best way to extend Jupyter Server? (e.g. like JupyterHub which spawns server subclasses with auth patched-in)?
  - Subclass ServerApp, not ExtensionApps.
  - To choose a specific frontend application as the default,
    1. enable the extension, e.g. put jpserver_extensions={"jupyterlab": True} somewhere.
    2. set the server applications default_url to the extension's endpoint, e.g. ServerApp.default_url="/lab"
Review Issues:
- Jupyter Server repo name: https://github.com/jupyter-server/jupyter_server/issues/356
  - Discussed not to change the name so that it matches the Python package name.
- Have web.authenticated be the default: https://github.com/jupyter-server/jupyter_server/issues/389
  - This might be too invasive of a change, but the principal is sound in general.
  - Alternative approach:
    - Add method in ServerApp that checks whether handlers are authenticated.
    - Add trait (and flag) in ServerApp to warn, exclude, modify, or allow handlers that are not authenticated. The default is allow, to keep backwards compatibility.

kevin-bates commented 3 years ago

February 4th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Vidar Fauske	JPMorgan	@vidartf
Zach Sailer	Apple	@Zsailer
Steve Silvester	Apple	@blink1073
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

Some mapping kernel manager issues:
- https://github.com/jupyter-server/jupyter_server/issues/402
- https://github.com/jupyter-server/jupyter_server/issues/403
Tests problems and questions
- Move tests module into the jupyter_server package?
- Cross package integration testing
Release Jupyter Server 1.3.0 today.
- Switch to AsyncKernelManager
- Improvements to ExtensionApp

Zsailer commented 3 years ago

February 11th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Pierre-Olivier Simonard	Quansight / ioapps	@pierrotsmnrd
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

file_to_run trait is broken: https://github.com/jupyterlab/jupyterlab/issues/8959
- ServerApp and ExtensionApp need a new trait, file_url_prefix.
  - this trait tells the server what url-prefix it should use to open a file directly.
  - nbclassic uses /notebook
  - jupyterlab uses /doc
- this uncovered some issues with file_to_run + root_dir. If these paths are incompatible, i.e. live on different branches of the filesystem, there is no helpful warnings/checks.
Content-based modification detection:
- https://github.com/jupyter-server/jupyter_server/issues/409
- Given adding a checksum to the model would be considered a version change, we felt it best to let this play out on the client (lab) side at this time.

Zsailer commented 3 years ago

February 18th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Zach Sailer	Apple	@ZSailer
Carol Willing	Noteable	willingc
Pierre-Olivier Simonard	ioapps.io / Quansight	@pierrotsmnrd
Steven Silvester	Apple	@blink1073
Sylvain Corlay	QuantStack	@SylvainCorlay
Eric Charles	Datalayer / Quansight	@echarles
Vidar T Fauske	JP Morgan Chase	@vidartf
Jason Grout	Bloomberg	@jasongrout
A. T. Darian	Two Sigma	@afshin
Jeremy Tuloup	QuantStack	@jtpio

Agenda + Minutes

Dependabot security alert on resource usage repo
- marked dependency [Should be removed from minutes ?]
- Jeremy: Should be fixed by: https://github.com/jupyter-server/jupyter-resource-usage/pull/87
jupyter_server_mathjax release :tada:
Open PRs:
- Tests moved into package: https://github.com/jupyter-server/jupyter_server/pull/416
  - Note https://github.com/pytest-dev/pytest/issues/4039
- Emit warning for load_jupyter_server_extension: https://github.com/jupyter-server/jupyter_server/pull/411
- Track only this server's kernels: https://github.com/jupyter-server/jupyter_server/pull/407
Jupyter Server architecture variants (looking ahead to RTC).
- Illustrations by @echarles

Zsailer commented 3 years ago

February 25th, 2021

Attendees

Name	affiliation	username
Vidar T Fauske	JP Morgan Chase	@vidartf
Zach Sailer	Apple	@Zsailer
Pierre-Olivier Simonard	ioapps.io / Quansight	@pierrotsmnrd
A. T. Darian	Two Sigma	@afshin
Mariko Wakabayashi	Twitter	@mwakaba2
Kevin Bates	IBM	@kevin-bates
Steven Silvester	Apple	@blink1073
Sylvain Corlay	QuantStack	@SylvainCorlay

Agenda + Minutes

Change translation function alias
- https://github.com/jupyter-server/jupyter_server/pull/424
- Question: will adding a deprecating warning raise a flood of warnings.
  - No, this is a 1-time warning.
- Merging away!
Unable to open a kernel if HTTP PATCH request is blocked by proxy
- https://github.com/jupyter-server/jupyter_server/issues/430
- Why was this not an issue in Jupyter Lab 2.x?
  - PATCH requests were still made when updating sessions in JupyterLab.
  - Let's track down this change in JLab and post it on the issue for reference.
- Falling back to GET/POST requests is a major change to Jupyter REST API
  - It would require a JEP.
  - Clients that want to support this change also need to handle this.
  - Is this common enough problem to warrant such a change?
    - For now, it makes the most sense to wait and see if this issue appears repeatedly before launching such a wide scale change.
jupyter/telemetry: nested category filtering
- https://github.com/jupyter/telemetry/pull/59
- Looking for reviewers on the Jupyter telemetry repository.
- Please reach out, or come to the weekly Jupyter Server meetings, if you're interested.

Zsailer commented 3 years ago

March 4th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf
Jeremy Tuloup	QuantStack	@jtpio
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

PATCH request change between JLab 2 and 3.
- https://github.com/jupyter-server/jupyter_server/issues/430
terminals excluded from activity monitoring #432
- Need to bring over a couple NB PRs
jupyter_server-related issues are showing up on notebook repo.
- We should port these to jupyter_server
- Here's an easy issue filter link for ServerApp.
- Can MeeseeksBot do the porting? Yup!

blink1073 commented 3 years ago

March 11th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Steven Silvester	Apple	@blink1073
Afshin Darian	Two Sigma	@afshin

Agenda + Minutes

Daylight savings time. Which TZ is the time linked to? :smiley:
Updated/updating the notebook port list
- https://github.com/jupyter-server/jupyter_server/issues/110
Improved automated publishing and packaging work in progress
- leverage Github Actions to:
  - generate a changelog
  - generate a (draft) Github release that contains the tar/wheel files
- changes to jupyter-packaging
  - thin layer on top of setuptools
  - new build backend that adds a pre-build step to pyproject.toml to build jupyter-like packages, i.e. includes an NPM builder before building the python package.
  - can use entrypoints instead of data_files
    - data_files still work, but we'll drop support for using data_files in develop mode.
    - static JS assets found using entrypoints

blink1073 commented 3 years ago

March 18th, 2021

Attendess

Name	affiliation	username
Steven Silvester	Apple	@blink1073
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

Current discussions for Realtime Collaboration on Authentication/Authorization across JupyerLab, Jupyter Server and JupyterHub
- Alpha design document for Authentication will be discussed at next RTC meeting on Monday (meeting details on https://github.com/jupyterlab/rtc)
- Same message will be posted at the today JupyterHub meeting (https://hackmd.io/YpwhVwbSSpCG19I0JNpqjQ)
Packaging PR: https://github.com/jupyter/jupyter-packaging/pull/69
- Should we offer specific functions?
  - wrap_cmdclass(pre_develop=None, pre_sdist=None, ....)
    - pre_develop, pre_sdist, pre_wheel
    - post_develop, post_sdist, post_wheel
Code Scan: https://github.com/jupyter-server/jupyter_server/issues/447
- PR: https://github.com/jupyter-server/jupyter_server/pull/449
- Set up on fork? https://docs.github.com/en/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning
Password check: https://github.com/jupyter-server/jupyter_server/issues/448
- Detect notebook password and use if server password is not set
- Show a warning if we use the notebook password
Notebook 6.3 - Go ahead an cut one - start with a changelog PR
Add schema for allowable API? https://github.com/jupyterlab/jupyterlab_server/issues/167
- In conjunction with a list of allowed server extensions, a schema for allowed routes sounds like a reasonable solution. This would allow an application to be secure without relying on an external service.

blink1073 commented 3 years ago

April 1st, 2021

Attendess

Name	affiliation	username
Steven Silvester	Apple	@blink1073
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin
Vidar T Fauske	JP Morgan Chase	@vidartf
Jeremy Tuloup	QuantStack	@jtpio
Sylvain Corlay	QuantStack	@SylvainCorlay

Agenda + Minutes

Issue: https://github.com/jupyter-server/jupyter_server/issues/455
- Will explore moving HTTPKernelManager and HTTPKernelClient implementations from Elyra to gateway package and let the current gateway (multi) kernel manager return instances of HTTPKernelManager (which will be renamed GatewayKernelManager)
- Vidar: Recently made KM for prewarmed kernels for Voila: https://github.com/voila-dashboards/hotpot_km/
Adopt a .github repo - in action in jlab
1.6 Release time

blink1073 commented 3 years ago

April 8th, 2021

Attendess

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Steven Silvester	Apple	@blink1073
Vidar T. Fauske	JP Morgan Chase	@vidartf
David Brochart	QuantStack	@davidbrochart
Zach Sailer	Apple	@Zsailer

Agenda + Minutes

Status of authorization PR 165, rebased via PR 467?
- David is going to update the original PR (165) with master and we'll try to keep this PR up to date
- Clients/applications (e.g., RTC) can then use this branch so we can get a feel for what additions are necessary
1.6.0 released - thanks Steve! Changelog produced using https://github.com/executablebooks/github-activity
Server event notification - https://github.com/jupyter-server/jupyter_server/issues/466
- Server-side event publisher with JSON schema backed events
- Possible events include kernelspec added/modified, file modified, kernel started/stopped, really any REST call other than GET. Could also add file system polling for contents events
- For the "server shutdown event", we would have a contract that consumers of the event should handle it synchronously
- Possible overlap with telemetry effort, might be efficient to share logic/infrastructure
- Short term solutions are to merge telemetry as a private implementation and use it for this emitter, or to create a trait on ServerApp that is the "status", and could be observed using trailets observer.

kevin-bates commented 3 years ago

April 15th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Steven Silvester	Apple	@blink1073
Vidar T. Fauske	JP Morgan Chase	@vidartf
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

1.6.1 release
jupyter packaging 0.9
handling of command line aliases
testing as downstream consumer in ipykernel
- We should also be testing downstream packages
[Kevin] Working on issue in new "nudge" logic when a kernel's restart results in port changes (part of Kernel Provisioning POC)

blink1073 commented 3 years ago

April 22nd, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf
Steven Silvester	Apple	@blink1073
A. T. Darian	Two Sigma	@afshin
Tony Fast	Quansight	@tonyfast

Agenda + Minutes

Release 1.6.4 is out - thanks Steve!
Refactor gateway kernel management (#483)
- Compatible with previous behavior
- Confirmed Papermill-level functionality using Elyra
- Kevin will rename GatewayKernelManagers to GatewayMappingKernelManager - preserving the current terminology
Anyio Version 3 is affecting AsyncContentsManagement (#487)
- Capped in last release
Reading/writing large notebooks
- Discuss ways to handle chunking notebook content on the server and providing an API (REST or Websocket) to clients to enable loading pieces of a notebook and not the whole notebook at once.
- Technologies discussed:
  - jsonlines functionality for returning notebook content with Tony
    - https://nbviewer.jupyter.org/gist/tonyfast/707ea716d9d413feef184e7bdbb67a93

Zsailer commented 3 years ago

April 29th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf
Steven Silvester	Apple	@blink1073
Kien Dang	National University of Singapore	@kiendang
A. T. Darian	Two Sigma	@afshin
Zach Sailer	Apple	@Zsailer

Agenda + Minutes

Jupyter Telemetry in Jupyter Server
- @kiendang has been doing great work pushing jupyter/telemetry forward
- jupyter/telemetry#60 will be merged later today.
- jupyter/telemetry#59 allows nested category filtering
  - Limitations are listed in the PR
  - This requires two validation steps: 1. to validate the event against its schema and 2. to extract categories.
  - This which might be a bottleneck.
  - For now, this might be okay. Since this is an opt-in feature, we can optimize later if its an issue.
  - Newer drafts of JSON schema include the concept of "annotations" that should address this problem anyways. We can switch to using annotations once that's formalized.
- Once the above 2 PRS are merged, we can finish up the jupyter_server PR, https://github.com/jupyter-server/jupyter_server/pull/364
  - Move the /api/eventlog endpoint into a separate PR for security reasons.
  - Need to enable some sort of token mechanism to verify that incoming events came from a trusted source.
Replacement/alternative to data files? - discussion
Jupyter Releaser coming soon :tada:

blink1073 commented 3 years ago

May 6th, 2021

Attendees

Name	affiliation	username
Max Klein	JP Morgan Chase	@telamonian
Steven Silvester	Apple	@blink1073
Vidar T Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Mariko Wakabayashi	Twitter	@mwakaba2
Kien Dang	NUS	@kiendang

Agenda + Minutes

Jupyter Releaser is released and we've used it to make two prereleases of server
Terminal testing on Windows
- Notebook tests are failing
- Server tests are skipped
- For terminado we only run the basic test
- jupyter_client has no terminal logic, but it does have Windows-specific logic we can emulate for shutting down processes.
Update on "Anyio version v3.0.0 breaks AsyncContentsManager" (#487)
- Fix: Upgraded anyio to version v3.0.1 for python3.7 and higher. (#492)
- New version is not working for python 3.6. The anyio team is looking into a fix for python 3.6. (#505)
Download folder as archive issue #68. Previously discussed in #251/#252.
Institute a process for adopting third party extensions similar to JupyterLab?
- We will add language to both our team compass and JuptyerLab team compass to point to jupyterlab-contrib
Regarding issue #496, Kevin will bump the dependency of jupyter_core to include the appropriate lower version

blink1073 commented 3 years ago

Jupyter Server Weekly Meeting

Where: jovyan Zoom (pwd: c0JZTHlNdS9Sek9vdzR3aTJ4SzFTQT09)
Historical notes can be found here.

May 13th, 2021

Attendees

Name	affiliation	username
Steven Silvester	Apple	@blink1073
Vidar Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Kien Dang	NUS	@kiendang

Agenda + Minutes

Added branch protection to multiple repos
1.7.0 released this week. There was one bug in the release process that was fixed in jupyter_releaser.

Zsailer commented 3 years ago

May 27th, 2021

Attendees

Name	affiliation	username
Nick Bollweg	Georgia Tech, deathbeds	@bollwyvl @nrbgt
Kevin Bates	IBM	@kevin-bates
Vidar Fauske	JP Morgan Chase	@vidartf
Zach Sailer	Apple	@Zsailer
Ryan C Cooper	UConn	@cooperrc
Mariko Wakabayashi	Twitter	@mwakaba2
Pierre-Olivier Simonard	Quansight	@pierrotsmnrd
Afshin T. Darian	Two Sigma	@afshin

Agenda + Minutes

Zach — back from parent leave at full-time. 🐣
PRs that need review still:
- extensions: allow extensions in namespace packages
  - https://github.com/jupyter-server/jupyter_server/pull/523
- stop hook for extensions
  - https://github.com/jupyter-server/jupyter_server/pull/526
Discussion: #518 - OpenAPI contents inaccuracies
- First problem—Jupyter Server's API spec is broken today. We need to fix it and spec validation.
- Second problem—the spec really isn't the standard today; rather, the reference implementation (i.e. jupyter_server) is the standard.
- High-level plan forward:
  - Move the OpenAPI spec in Jupyter Server to a separate "specs" repository.
  - Make this spec true to the current Jupyter Server implementation.
  - Over time, begin making the spec more well-defined.
  - Jupyter Server, then, follows this spec as the reference implementation.
  - Should it go in the JEP?
    - JEP isn't really a good place for standards.
- Practical plan forward:
  - Update the current spec in-place to match Server
  - Possibly create a temporary "spec" repo in Jupyter Server?
  - https://discourse.jupyter.org/t/sustainability-of-the-ipynb-nbformat-document-format/3051
  - https://xmpp.org/extensions/

Zsailer commented 3 years ago

June 3rd, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Afshin T. Darian	Two Sigma	@afshin
Tony Fast	Quansight	@tonyfast
Pierre-Olivier Simonard	Quansight	@pierrotsmnrd
Max Klein	JP Morgan Chase	@telamonian

Agenda + Minutes

Running tasks in Jupyter Server's ioloop at start-up.
- https://github.com/jupyter-server/jupyter_server/pull/531
- Set the io_loop attribute earlier in ServerApp lifecycle.
- If there are no concerns with allowing extensions to run tasks in the server io_loop, this PR can be merged once updated.
Python namespace packages with extensions
- https://github.com/jupyter-server/jupyter_server/pull/523
- Checking if __file__ is in the package root might not be the most robust way to identify namespace packages.
- Tony will post some ideas on using importlib versus __import__
  - one might be able to inspect the importlib object to see which loader (i.e. NamespaceLoader) was used to load the package.
Access files outside of notebook-dir (root-dir)
- https://github.com/jupyter-server/jupyter_server/issues/534
- Approach:
  - Add a new trait to represent the "preferred" or "starting" directory (e.g., preferred_dir)
  - preferred_dir
    - will default to root_dir (aka notebook_dir)
    - is required to be an existing directory residing at or within root_dir
    - is a "suggestion" to client applications. Client applications can honor preferred_dir and start their file-browser at preferred_dir instead of root_dir.
Support various config in jupyter_server_config.d/*.json
- eg: https://github.com/jpmorganchase/jupyter-fs/pull/108#issuecomment-853438120
```
{
  "ServerApp": {
    "contents_manager_class": "jupyterfs.metamanager.MetaManager",
    "jpserver_extensions": {
      "jupyterfs.extension": true
    }
  }
}
```
- Talked about previously, but unresolved in issue #207
- Jupyter Server only looks for jpserver_extensions trait in the static config dir (conf.d), since this dir was originally intended for discovering extensions.
- Open question: should we allow extensions to affect other server traits in this static config?
  - How do we handle conflict? —just load alphabetically and raise a warning in --debug.
  - User config supersedes static config, so users can manually override anything set here if there's conflict.
- Let's continue the conversation in #207.

Zsailer commented 3 years ago

June 17th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf
David Brochart	QuantStack	@davidbrochart
Zach Sailer	Apple	@Zsailer

Agenda + Minutes

Authorization in request handlers (#165)
- Open question: what is the right permissions model:
  - read, write, execute? (possibly) delete?
  - HTTP verbs? (i.e. GET, POST, PATCH, PUT, DELETE, WEBSOCKETS)
  - Decision: stick with read, write, execute.
    - The HTTP request object is passed to the authorization hook.
    - If more granularity is needed, one could inspect the object for request type and apply whatever filtering they need.
- Authorization hook should be set by a trait (currently, it's a monkeypatch).
- Open question: should the authorization hook be a function or class.
  - Decision: it should be a class—following the pattern set in Jupyter Server.
  - JS should provide an ABC (AuthorizationManagerABC) for others to extend and a default, concrete implementation.
    - The default implementation (AuthorizaionManager?) always returns True, i.e. no authorization enforced.
  - The authorization manager will be specified by a trait named authorization_manager_class with a required klass of AuthorizationManagerABC and defaulted to AuthorizationManager.
- Future discussion that's needed:
  - Do we need finer-grained permissions on Kernel messages on the ZMQ channels?
  - Fortunately, this would be an additive change and we can handle in a later PR.
RBAC in Jupyter Server (#540)
- This PR defines scopes and roles for Jupyter Server.
- RBAC is an implementation of the authorization layer mentioned above.
- This work should be a separate extension under the jupyter-server org.

blink1073 commented 3 years ago

June 24th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Steven Silvester	Apple	@blink1073
Afshin T. Darian	Two Sigma	@afshin

Agenda + Minutes

Will need to cut a release Fix nbconvert handler to fix shutdown behavior in JupyterLab - We're due for a 1.9 release
Released v1.9.0
Discussed health-check issue from Notebook
- Darian commented that hitting /api already returns the server's version info and is not authenticated - which is sufficient.
Question about extending an app in a script (currently not possible due to insistence that extension apps be full extensions), will create repro draft PR.

Zsailer commented 3 years ago

July 1st, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Zach Sailer	Apple	@Zsailer
Afshin T. Darian	Two Sigma	@afshin

Agenda + Minutes

Team-compass repo https://github.com/jupyter-server/team-compass
- start a readthedocs page, borrowing from jupyterhub.
- Following https://github.com/jupyter/governance/pull/98, bootstrap a team over the next couple weeks.
https://github.com/jupyter-server/jupyter_server/pull/526 and/or #547
- We discussed whether this necessitates a major release since it changes API.
  - Turns three synchronous methods into async methods.
- Concluded it's very unlikely that others have overridden these methods.
- Decision:
  - Merge PR this week.
  - Post a comment to make others aware of the API change.
  - Wait a week to make a minor release.
https://github.com/jupyter-server/jupyter_server/pull/523
- Currently, PR relies on private APIs, which is concerning.
- Can we build a more "defensive" way to discover namespace packages?
  - Make get_namespace_package try a series of steps to find a NS package
    1. Try importing the private standard lib API to inspect the namespace package (how it's currently implemented in the PR)
    2. If those private APIs are not found (i.e. change in the future), fallback to trying to import the namespace package directly.
    3. If that fails, bail gracefully.

Zsailer commented 3 years ago

July 15th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Vidar Fauske	JP Morgan Chase	@vidartf
Afshin T. Darian	Two Sigma	@afshin
Eric Charles	Datalayer, Quansight	@echarles
Oliver Sanders	Met Office	@oliver-sanders
Max Klein	JP Morgan Chase	@telamonian

Agenda + Minutes

Kernel Specification Manager JupyterLab Extension
- WIP of POC https://github.com/Quansight/ksmm
- in relation with
- Kernel Provisioner: https://github.com/jupyter/jupyter_client/pull/612
Jupyter Server authorization update
Serve Directories through ServerApp.extra_static_paths
- https://github.com/jupyter-server/jupyter_server/issues/551
Setting extra traitlets of a LabServerApp sub class via the command line doesn't work since v1.6.4.
- https://github.com/jupyter-server/jupyter_server/issues/550
1.10 release?

Zsailer commented 3 years ago

July 22th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Vidar	JP Morgan Chase	@vidartf
A. T. Darian	Two Sigma	@afshin
Andrii Ieroshenko	Cal Poly Jupyter	@andrii-i

Agenda + Minutes

Releasing 1.10.0 happening today.
Please review the Authorization PR
Questions:
- Which type of API server should we choose, GraphQL or REST?
  - Jupyter Cal Poly intern project, Notifications Extension
  - Depends on the types of queries the client will need to make to the server.
  - Early on, it might be easier to develop the API server using REST, since the queries are relatively simple.
- What's the "jupyter-supported" way to add configuration (not using settings) to a JupyterLab extension.
  - Please see https://jupyter-server.readthedocs.io/en/latest/operators/configuring-extensions.html and https://jupyter-server.readthedocs.io/en/latest/developers/extensions.html#authoring-a-configurable-extension-application
  - Adam plans on putting together an example once he has it working.

Zsailer commented 3 years ago

July 29th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Eric Charles	Datalayer	@echarles
Rick Wagner	UCSD	@rpwagner
Rollin Thomas	NERSC	@rcthomas
Steven Silvester	Apple	@blink1073
Mariko Wakabayashi	Twitter	@mwakaba2
Vidar T Fauske	JP Morgan Chase	@vidartf
Gonzalo Peña-C	Quansight, inc	@goanpeca
Max Klein	JPM	@telamonian

Agenda + Minutes

Security documentation and workshop
- https://jupyter-server.readthedocs.io/en/latest/operators/security.html
- https://jupyter-notebook.readthedocs.io/en/stable/security.html
- https://jupyterhub.readthedocs.io/en/stable/reference/websecurity.html
- https://www.cvedetails.com/vulnerability-list/vendor_id-15653/Jupyter.html
- Best Practices badge
- Also: Trusted CI engagement, supporting workshop and beyond
- Engagements from companies may be an opportunity
- Preparation effort includes interviews with people doing various deployments in different contexts
- Possible overlap w/telemetry workshop
Demo (from @echarles)
- Multi-user, GitHub authenticated, OPA-based (Open Policy Agent) authorization (based on JWT token)
- Uses authorization PR
  - It appears we're not always returning the proper status codes when an action is unauthorized (i.e. 401 vs. 403).
- Uses Tornado sessions
- JupyterLab doesn't gracefully handle 401/403 responses. This likely requires a PR to JupyterLab.
- Can we query the server for a users permissions before trying to make a request?
  - Previously, there was talk about adding an "identity" extension/endpoint to Jupyter Server.
    - This endpoint would return user info.
Performance analysis and improvements:
- Meeting is happening next Wednesday
- https://discourse.jupyter.org/t/performance-analysis-and-improvements/10126

Zsailer commented 3 years ago

August 5th, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Steven Silvester	Apple	@blink1073
Vidar Fauske	JPMC	@vidartf
A. T. Darian	Two Sigma	@afshin
Eric Charles	Datalayer	@echarles
Jessica Xu	Quansight	@jess-x
Zach Sailer	Apple	@Zsailer

Agenda + Minutes

Look and consider:
- https://github.com/jupyter-server/jupyter_server/issues/565
  - Answering Oliver's question, yes—this should definitely be a "feature" in Jupyter Server.
  - We could set our own subclassed loaders on the python_config_loader_class class attribute (and potentially the json on) on our Application object. That change to the loader(s) can then be upstreamed to traitlets after.
- https://github.com/jupyter-server/jupyter_server/issues/566
Next steps for Parameterized Kernels?
- Let's use a 15-minutes chunk of Jupyter Server meetings each week to discuss this topic.
- Can hopefully reuse the settings editor for Lab for UI for editing parameters (JSON schema -> form)
Add a Security Policy for Server?
- Template includes supported versions and how to handle a vulnerability report - we could link to our security docs on RTD for now until jupyter.org/security is available

Zsailer commented 3 years ago

August 12th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin
Steven Silvester	Apple	@blink1073
Rollin Thomas	NERSC	@rcthomas
Jessica Xu	Quansight	@jess-x
Oliver Sanders	Met Office	@oliver-sanders
Vidar T Fauske	JPMorgan Chase	@vidartf

Agenda + Minutes

Issues to review:
- Add a setting to allow non-empty directory deletion on windows. https://github.com/jupyter-server/jupyter_server/issues/570
  - Background: https://github.com/jupyter/notebook/issues/3631
  - Current Status: https://github.com/arsenetar/send2trash/issues/28
    - send2trash hasn't resolved this issue on Windows, yet.
  - Windows specific setting.
  - If we add this setting, what should be the default?
    - Right now, we have different behavior depending on OS.
    - default=True makes all systems behave the same.
    - default=False keeps the system as-is.
  - If send2trash fixes this issue, this setting/trait wouldn't be needed.
    - We'd alway allow deletion of non-empty directories.
    - The trait would enter deprecation cycle.
- https://github.com/jupyter-server/jupyter_server/issues/566
  - We'll revisit this issue at next week's meeting.
Monitor kernel process data per kernel (remote kernel)?
- https://github.com/jupyter-server/jupyter-resource-usage/issues/58#issuecomment-654905584
- Add plumbing to Jupyter Server to allow per-kernel monitoring.
  - Add a /api/kernels/<kernel-id>/metrics endpoint to each kernel.
- No one is actively working on this today.
- Kernel Provisioner: https://github.com/jupyter/jupyter_client/pull/612
Open Telemetry: https://opentelemetry.io/
- Exploring OpenTelemetry.
- Has a Tornado plugin for logging traces from Tornado handlers.
- Possible backend for Jupyter Telemetry.

Zsailer commented 3 years ago

August 19th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin
Steven Silvester	Apple	@blink1073
Vidar T Fauske	JPMorgan Chase	@vidartf
Eric Charles	Datalayer	@echarles
Max Klein	JPM	@telamonian
Jessica Xu	Quansight	@jess-x

Agenda + Minutes

Issues to review:
- https://github.com/jupyter-server/jupyter_server/issues/566
  - Close this issue per Vidar's comments on the issue.
terminal test failing on Windows : https://github.com/jupyter-server/jupyter_server/pull/571
- increasing the timeout seems to fix the issue.
nbconvert performance regression
- This causes a major slow-down in Jupyter Server; need to investigate if we can fix in NBConvert and/or Server.
- Original regression identified here: https://github.com/jupyter/nbconvert/issues/1439
- Fix for notebook use here: https://github.com/jupyter/nbconvert/pull/1471
- We also call the get_exporter, so fix does not solve for us: https://github.com/jupyter-server/jupyter_server/blob/1444737514c3e8c6acbe3fc01f9420c8e078d3a0/jupyter_server/services/nbconvert/handlers.py#L27
Parametrized Kernel JEP
- POC in 3 branches https://github.com/jupyter/enhancement-proposals/pull/46#issuecomment-898357256
  - jupyter client: https://github.com/datalayer-externals/jupyter-kernel-client/tree/exp/kernel-params
  - jupyter server: https://github.com/datalayer-externals/jupyter-server/tree/exp/kernel-params
  - jupyterlab: https://github.com/datalayer-externals/jupyterlab/tree/exp/kernel-params
- Updated proposal to be based on provisioners
What happened to kernel stopping in jupyter_client, and why were a few versions yanked from pypi?
- related:
- jupyter_client#618
- jupyter_client#620
- specifically, what is killpg supposed to do?
- When using enterprise_gateway, jupyter_client v6.2.0 interrupts the gateway instead of the kernel on kernel stop request
- ~~temp~~ fix: downgrade to 6.1.12
Meeting later today to discuss Jupyter Resource Usage
- Meeting 30 minutes after this meeting.
- Google Meet: https://meet.google.com/dth-kprw-jfv?authuser=0

Zsailer commented 3 years ago

August 26th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin
Max Klein	JPM	@telamonian
Karla Spuldaro	IBM	@karlaspuldaro

Agenda + Minutes

Issues to review:
- How best to get rid of GatewayClient.env_whitelist?
  - https://github.com/jupyter/enterprise_gateway/pull/1000#issuecomment-903301465
  - Does it make sense to determine the env vars that get forwarded from user server -> gateway -> kernel by examining the env stanza of a kernelspec?
    - pro: kernelspec should behave similarly with/without a gateway
    - con: is it more or less confusing this way?
Jupyter Resource Usage
- discussed roadmap/potential requirements last week
- extension demo next week
- requirements in-progress
Long term, re-occuring topics:
- There a few larger efforts on the horizon for Jupyter Server.
- We should have a place where we post documentation about these efforts (separate repo?).
- Multi-user server
  - identity, storage (DB), auth ...
- Kernel parameterization
  - Current draft proposal

kevin-bates commented 3 years ago

September 2nd, 2021

Attendees

Name	affiliation	username
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf
Karla Spuldaro	IBM	@karlaspuldaro
Jessica Xu	Quansight	@jess-x
A. T. Darian	Two Sigma	@afshin
Max Klein	JPM	@telamonian
Mariko Wakabayashi	Twitter	@mwakaba2
Thi Nguyen	Stealth-mode startup	@duongnt

Agenda + Minutes

Darian: What does a Jupyter server data layer look like?
- By "data layer", we are talking about things like user settings, workspaces, event logs, etc. that would be a natural fit for a SQL database
- Discusssion Notes
  - Use cases
    - Multi-user environment where users want to track own settings.
    - extensions like jupyterlab-notifications to store user specific notification data
  - Possible storage solutions: Key Value Store vs Sqlite vs SQL
  - Prevent X individual extensions to connect to X different databases to store configuration etc.
  - Use standard sqlite version?
  - Define some kind of standard on the Jupyter level for the database schema to ensure the tables can be joined with Jupyter server maintained tables (e.g. user, logging, events, metrics)
  - Who would work on this? (e.g. jupyterlab, jupyter_server)
- Conclusion (for now) We should first pursue a storage solution for the known data that we plan to offer in a multi-user server:
- user-specific settings
- user-specific workspaces
- event log
- real-time collaboration CRDT store

Zsailer commented 3 years ago

September 9th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Luciano Resende	IBM	@lresende
Thi Nguyen	Startup	@duongnt
David Brochart	QuantStack	@davidbrochart
Karla Spuldaro	IBM	@karlaspuldaro
Adrien Delsalle	QuantStack	@adriendelsalle
Kevin Bates	IBM	@kevin-bates
Steven Silvester	Apple	@blink1073
A. T. Darian	Two Sigma	@afshin

Agenda + Minutes

New Release: v1.11.0 (thanks, Steve!)
Proposal: Towards a new generic and composable server
- https://github.com/jupyter-server/team-compass/issues/11
- Should the following repos move into the Jupyter Server organization?
  - https://github.com/adriendelsalle/fps
  - https://github.com/davidbrochart/jupyverse
  - If so, we'll need to have clear messaging that says these projects are experimental.
  - We're not abandoning the current Jupyter Server :smile:
  - We'll vote on this next week.
- Technology stack overview:
  - Pydantic and Typer:
    - Embrace Python's native type hints/annotation.
    - Together, they remove most of the need for Traitlets
    - Handles most data validation automatically, and offers a simple way to define custom types.
    - Typer offers a simple API for adding command-line interfaces to applications for configuring Python objects.
    - Can we drop traitlets? What's left in Traitlets?
      - “Observe” other traits. Necessary for ipywidgets.
      - loading configuration from file.
        
        Comparatively, this is a small amount of code to maintain.
        
        We can likely contribute these things upstream too.
  - FastAPI
    - Built on top of Pydantic.
    - Native syntax for type annotations/hints
    - Auto generate (developer) documentation and unit-tests (via hypothesis) for free.
    - ASGI implementation, i.e. async by default.
  - Pluggy
    - Provides a significantly more robust mechanism for discovering and loading application extensions.
    - As many people have experienced, today’s Jupyter Server's extension discovery/loading is quite brittle.
- Questions:
  - Does this change the kernel messagine spec? Will this invalidate current kernels?
    - No.
  - Will server extensions have to migrate?
    - Yes.
    - Is it possible to somehow prevent or minimize this in the future?
      - Today's server extension API isn't (and was never) clearly specified. Pluggy defines a clear specification for plugin systems. It's more stable and less likely to change in the future.
  - Will this change the current REST API?
    - It's not a immediately clear. Right now, no. It's possible this new server implementation would expose needed changes in the current REST API, but that would require a JEP.
  - Does this need to wait for classic Notebook to retire?
    - This is an issue to be addressed by the new software steering council. Yes, this should coincide the the retirement of the classic notebook server.

Zsailer commented 3 years ago

September 16th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Adrien Delsalle	QuantStack	@adriendelsalle
Raman Tehlan	Raman Tehlan	@ramantehlan
Sylvain Corlay	QuantStack	@SylvainCorlay
David Brochart	QuantStack	@davidbrochart
Vidar T Fauske	JP Morgan Chase	@vidartf
Thi Nguyen	Startup	@duongnt
Karla Spuldaro	IBM	@karlaspuldaro
Jeremy Tuloup	QuantStack	@jtpio
Steven Silvester
Vidar Tonas Fauske
Adam Patterson
Corey Smith
Kevin Bates	IBM	@kevin-bates

Agenda + Minutes

[VOTE] Move the following repos into the Jupyter Server organization:
- Repos to move:
  - https://github.com/adriendelsalle/fps
  - https://github.com/davidbrochart/jupyverse
- This simply makes it easier to collaborate on these projects while they are experimental and being developed.
- It does not mean that the Jupyter Server team is adopting the proposed server, yet. We'll have another vote at a later time for the adoption of this work.
- If so, we'll need to have clear messaging that says these projects are experimental.
- [APPROVED] David and Adrien will move these repos to Jupyter Server org sometime over the next couple of weeks.
Thought about the backend extension system brought up...
- Enable each server extension runs its own server and exposes a port.
- connect to that extension's port using jupyter-server-proxy.
- This works for simpler server extensions that aren't coupled to one another or don't use any of the managers, but more complex extensions that need access to the core managers can't use this approach.
- A major thing to keep in mind— auth, loginhandler, etc. needs to be feature complete. We don't want to make multiple requests to the auth provider per request.

kevin-bates commented 2 years ago

September 16th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Adrien Delsalle	QuantStack	@adriendelsalle
Raman Tehlan	Raman Tehlan	@ramantehlan
Sylvain Corlay	QuantStack	@SylvainCorlay
David Brochart	QuantStack	@davidbrochart
Vidar T Fauske	JP Morgan Chase	@vidartf
Thi Nguyen	Startup	@duongnt
Karla Spuldaro	IBM	@karlaspuldaro
Jeremy Tuloup	QuantStack	@jtpio
Steven Silvester
Vidar Tonas Fauske
Adam Patterson
Corey Smith
Kevin Bates	IBM	@kevin-bates

Agenda + Minutes

[VOTE] Move the following repos into the Jupyter Server organization:
- Repos to move:
  - https://github.com/adriendelsalle/fps
  - https://github.com/davidbrochart/jupyverse
- This simply makes it easier to collaborate on these projects while they are experimental and being developed.
- It does not mean that the Jupyter Server team is adopting the proposed server, yet. We'll have another vote at a later time for the adoption of this work.
- If so, we'll need to have clear messaging that says these projects are experimental.
Thought about the backend extension system brought up...
- Enable each server extension runs its own server and exposes a port.
- connect to that extension's port using jupyter-server-proxy.
- This works for simpler server extensions that aren't coupled
- A major thing to keep in mind— auth, loginhandler, etc. needs to be feature complete. We don't want to make multiple requests to the auth provider per request.

Zsailer commented 2 years ago

September 23th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Karla Spuldaro	IBM	@karlaspuldaro
Jeremy Tuloup	QuantStack	@jtpio
Kevin Bates	IBM	@kevin-bates
Adrien Delsalle	QuantStack	@adriendelsalle

Agenda + Minutes

No planned discussion in the notes.
Open discussion around Jupyverse
- Can Jupyverse work with Jupyter Server's managers?
  - It would make it easier for people to transition to this new server if the old managers worked.
    - The benefit of the old managers is that you can subclass those classes, borrowing most of their logic and only replacing the things you need.
    - Multi-managers are critical. We might need multimanagers in every service, i.e. contents, kernels etc. They handle logic that applies to all single managers, e.g. kernel culling.
  - This would be a good opportunity to refactor the managers and possibly replace traitlets for something like Pydantic and fps.config.
- How do we handle plugins where we don't want to include all the routes?
  - Today, we can loop through the Tornado handlers and remove (via monkeypatch) handlers that we don't want.
  - In JupyterLab plugins, you can disable portions of a plugin we don't want to used.
  - In Jupyverse, we can adopt a similar model. Add a way to disable parts of a plugin.
  - Jupyverse will not allow two plugins to have the same route (today, this would silently happen in tornado). If they do, you must configure your plugins to disable one of the duplicate plugin routes.
- How will Jupyverse handle services that are coupled, like today's sessions service in Jupyter Server?
  - Plugins can depend on other plugins in jupyverse/fps via a dependency injection system.
- Identity is already being handled in jupyverse (although, it's in a very early stage). See this new endpoint for listing users. Already moving in the direction of multiuser server!

Zsailer commented 2 years ago

September 30th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Rick Wagner	UCSD	@rpwagner
Adrien Delsalle	QuantStack	@adriendelsalle
David Brochart	QuantStack	@davidbrochart
Karla Spuldaro	IBM	@karlaspuldaro
Kevin Bates	IBM	@kevin-bates
Sylvain Corlay	QuantStack	@SylvainCorlay

Agenda + Minutes

Jupyter security-related documentation survey by Trusted CI @rpwagner
- Please review and add comments.
- This document is helpful in identifying gaps in the Jupyter Server docs.
- Thank you, Kay Avila, for compiling this survey!
FPS demo, shown by @adriendelsalle (Fast Pluggable Server)
- Showed the "Hello World" plugin
- Showed hooks for registering custom exceptions, configuration, and server routes.
- Showed autodocumentation generated by FastAPI.
- (Briefly) showed a full Jupyverse server built on FPS.
- Jupyverse conversation
  - This is a good time to start trying out the new server.
  - There are questions that need answering soon around auth
    - Can we make an opinionated decision and choose OIDC as the only supported auth protocol? This enables us make opinionated decisions moving forward for RTC, etc.

kevin-bates commented 2 years ago

October 7th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Mariko Wakabayashi	Twitter	@mwakaba2
Vidar Tonaas Fauske	JP Morgan chase	@vidartf
David Brochart	QuantStack	@davidbrochart

Agenda + Minutes

https://github.com/jupyter-server/jupyter_server/issues/585
- The jp_serverapp fixture is clearing all logging handlers on the main logger
- This prevents us from testing the output from the default logger
- The default logging handler is a StreamHandler that points at stdout/stderr. This gets cleared by pytest.
- https://github.com/jupyter-server/jupyter_server/pull/588
  - Reroutes the logs to a different stream and prints the results at the end of the test.
Issue in nbclassic that's breaking nbextensions_configurator
- The order of the handlers is different between classic notebook and nbclassic
- To address this, we'll have to inject the handlers in the right place.
Mariko: my team at Twitter is working on a blog post to share how Twitter uses Jupyterlab internally - will share blog post link when that gets published (around 10/30th)

Zsailer commented 2 years ago

October 14th, 2021

Attendees

Name	affiliation	username
Vidar Tonaas Fauske	JP Morgan chase	@vidartf
Steve Silvester	Apple	@blink1073
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Carlos Herrero	QuantStack	@hbcarlos
Jeremy Tuloup	QuantStack	@jtpio

Agenda + Minutes

Slow Starting Kernels Proposal (We'll convert this to an issue for further discussion)
- Problem: Jupyter Notebook was originally built with the assumption that kernels would start quickly. This turns out to not be true for some local kernels and most remote kernels.
- We previously proposed changing the REST API to reflect kernels/sessions that were "pending". The downside to a REST API change is that the server would need to advertise capability through a versioned API or some other status, and clients would need to be updated to accommodate the changes.
- An alternative method is to leave the current REST APIs intact and instead introduce the concept of a "pending" kernel that acts like a regular kernel from the client's perspective.
- A POST to /api/sessions or /api/kernels would create a "pending" kernel and return immediately before starting the kernel.
- It remains to be seen during implementation what changes need to be made to handlers and managers, but at the very least we will use a scheduled callback to actually start the kernels when we are handling the POST. The MappingKernelManager will also need to be updated to handle pending kernels internally in its public methods.
- We should use the kernel manager to get the kernel id
- We need to think about how kernel failure to start is handled for the user. Previously, it could be given to the user in the response to a POST
- We might even be able to add the pending logic to the handlers without needing to affect the managers (e.g by calling save_state on the managers directly)

Zsailer commented 2 years ago

October 21st, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Jessica Xu	Quansight	@jess-x

Agenda + Minutes

Custom messages across the kernel control channel
- Inspired by: https://github.com/jupyterlab/jupyterlab/pull/11285
- Can we generalize the control channel to allow custom messages?
- Use a schema-based approach.
  - Register schema with the kernel
  - Messages are instances of schemas and validated against registered schemas.

Zsailer commented 2 years ago

October 28th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Adrien Delsalle	QuantStack	@adriendelsalle
Steve Silvester	Apple	@blink1073
Jason Weill	AWS	@jweill-aws
Karla Spuldaro	IBM	@karlaspuldaro
Piyush Jain	AWS	@3coins
Jessica Xu	Quansight	@jess-x

Agenda + Minutes

NBClassic 0.3.4 released
Restart kernels in-place: https://github.com/jupyter-server/jupyter_server/issues/596
Slow starting kernels is becoming a common theme:
- https://github.com/jupyter/jupyter_client/issues/715
- https://github.com/jupyter-server/jupyter_server/issues/592
- Consider proposing a virtual community workshop on slow starting kernels.
Pending Kernels Status
- Jupyter client: https://github.com/jupyter/jupyter_client/pull/712
- jupyter server: https://github.com/jupyter-server/jupyter_server/pull/593
- jupyterlab: https://github.com/jupyterlab/jupyterlab/pull/11358
- Both jupyter_client and jupyter_server changes are ready for review, merge, release.
  - This feature is opt-in
  - Only requres a minor release—low risk change, since this feature is opt-in
- A decision made on the PR (open for comment though): What should happen when a kernel is restarted while it's in a pending state?
  - Don't allow a restart.
  - Instead, raise an error that's distinct to this scenario.

Zsailer commented 2 years ago

November 4, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Karla Spuldaro	IBM	@karlaspuldaro
Jason Weill	AWS	@jweill-aws
Kevin Bates	IBM	@kevin-bates
Mariko Wakabayashi	Twitter	@mwakaba2
Steven Silvester	Apple	@blink1073
Jeremy Tuloup	QuantStack	@jtpio
Jessica Xu	Quansight	@jess-x

Agenda + Minutes

Jupyter Server v1.11.2 released this week.
https://github.com/jupyter-server/jupyter_server/issues/605
- Assigned!
Pending Kernels
- All three PRs are ready, but we can't really try it out until all three are merged and available. Question about whether the JupyterLab PR can be released as a patch release (also raised the question yesterday at the Lab meeting).
- We decided that the JupyterLab PR could be merged and released in an alpha so we can test it and consider whether to backport it.
- We also questioned what the behavior would be in nbclassic with pending kernels. After some testing, the behavior with a pending kernel is not as seamless as a non-pending kernel in terms of the UI interactions, but all of the lifecycle methods work: starting, changing kernels, restarting, shutting down.
[Discussion] Telemetry in server
- Doing some initial explorations with auto-instrumenting Python classes in jupyter_telemetry (PR posted soon).
- Goals for jupyter-server telemetry:
  - Straight forward configuration
  - Uninvasive in the source code
  - Well defined JSON schemas
  - Opt-in
- What's the right level of granularity?
  - Schemas should be defined at the (Python) class level
    - This makes telemetry configuration straight-forward.
    - "Turn on" telemetry at each class level.
  - Individual events are collected at the function/method level.
- Current progress:
  - record decorator that wraps a method and emits:
```
{
  "method": ...,
  "args": ...,
  "kwargs": ...,
  "returned": ...,
  "time": ...
}
```
  - Instrument metaclass in Python
    - Decorate (listed) methods with a record wrapper than emits an event.
    - Comment: metaclasses can cause all kinds of issues, particularly with ABCs. Verify this won't break things for traitlets objects, etc.
  - Instrument the APIHandler in jupyter_server
    - log all get, post, patch, put, and delete methods.
    - Defined JSON schemas for each service.
- Expect to see PRs (in jupyter_server, and jupyter_telemetry) by next week.

Zsailer commented 2 years ago

November 11th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Vidar T Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Piyush Jain	AWS	@3coins

Agenda + Minutes

From this week:
- Shoutout :tada:, @mwakaba2! Thank you! https://github.com/jupyter-server/jupyter_server/pull/607
- Good first issue: https://github.com/jupyter-server/jupyter_server/issues/606
- https://github.com/jupyter-server/jupyter_server/issues/609
  - More gracefully handle when an io_loop doesn't exist.
  - A sigterm can come in at any time, even during startup.
  - This is an issue with classic notebook server, not jupyter-server. It's fixed in jupyter-server. (https://github.com/jupyter-server/jupyter_server/blob/a1b013ef76304193c059866e287d67cbfbdef4c5/jupyter_server/serverapp.py#L2671-L2682)
Pending Kernels
- JupyterLab v4.0.0a15 released
- Testing steps:
  - Checkout https://github.com/jupyter/jupyter_client/pull/712
  - Checkout https://github.com/jupyter-server/jupyter_server/pull/593
  - Install JupyterLab 4 alpha, pip install jupyterlab --pre
Persistent Session Managers
- The session manager is not configurable or persistent.
  - Session data is stored in a in-memory SQLite database, and cannot be configured to write to disk without subclassing.
  - Also, the session manager never calls .commit(). If the session manager is subclassed and written to disk, all methods must be overwritten/wrapped to call .commit() to actually store the changes.
- Proposed changes:
  - Make database path a configurable trait
  - Call .commit() after every method.
- Verify that .commit() is no-op in :memory: mode for the above changes to work.
Time for a Jupyter Server 2.0. Discuss timelines next week.
PR to review in Jupyter telemetry: https://github.com/jupyter/telemetry/pull/67

Zsailer commented 2 years ago

November 18th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Piyush Jain	AWS	@3coins
Karla Spuldaro	IBM	@karlaspuldaro
Mariko Wakabayashi	OpenZeppelin	@mwakaba2
Jason Weill	AWS	@jweill-aws
Rick Wagner	UCSD	@rpwagner
Steven Silvester	N/A	@blink1073
Sylvain Corlay	QuantStacck	@SylvainCorlay

Agenda + Minutes

Insights from this week:
- Issues:
  - https://github.com/jupyter-server/jupyter_server/issues/606
  - https://github.com/jupyter-server/jupyter_server/issues/609
- Pull requests:
Jupyter Server Team
- Software steering council updates.
Jupyter Security update
- Notebook Server Security Doc
Pending Kernels - any feedback?
PR Script for Maintainers (e.g. run pre-commit on this PR)
Base Setup Action - allows us to have a common setup for workflows across repos

Zsailer commented 2 years ago

December 2nd, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
David Brochart	Quantstack	@davidbrochart
Steve Silvester	MongoDB	@blink1073

Agenda + Minutes

https://github.com/jupyter-server/jupyter_server/pull/624
- If y-py is found at runtime, pull contents from the server-side (y-py) document model that's syncing (i.e. as a peer) of the Y.js patch server on "save".
- Question: what does "save" mean when RTC is enabled?
  - In Google Docs, there is not "save" feature. The document is automatically saved on intervals. The RTC is always keeping the latest version of the document
  - Checkpoints become useful. They are previous versions of the document.
  - The "save" API should probably pass-through in an RTC context.
- Alternative to this PR—create a custom ContentsManager in JupyterLab server that is applied when collaborative=True.

Zsailer commented 2 years ago

December 9th, 2021

Attendees

Name	affiliation	username
Vidar T Fauske	JP Morgan Chase	@vidartf
Jason Weill	AWS	@jweill-aws
Brian Granger	AWS	@ellisonbg
Rahul Goyal	AWS	@rahul26goyal
Zach Sailer	Apple	@Zsailer
Mariko Wakabayashi	OpenZeppelin	@mwakaba2
Karla Spuldaro	IBM	@karlaspuldaro
Kevin Bates	IBM	@kevin-bates
Carlos Herrero	QuantStack	@hbcarlos

Agenda + Minutes

Insights from the week
- Released Jupyter Server 1.13.0
- Merged PRs:
- New PRs:
  - https://github.com/jupyter-server/jupyter_server/pull/634
  - https://github.com/jupyter-server/jupyter_server/pull/635
- New Issues:
  - https://github.com/jupyter-server/jupyter_server/issues/633
  - https://github.com/jupyter-server/jupyter_server/issues/627
Timeout on starting kernels of R and Julia
- This breaks other kernels. We should fix this and cut a patch release ASAP.
- We should nudge both the control and shell channels.
- Zach will work on this today.
Memory leak detected with "remote" kernels
- Jupyter Server solution for this should take pending kernels into account.
Terminal Idle State PR
- Should this be in a contrib repo? (we don't currently have one). It's marked as "experimental".
- Maybe it's time to make terminals into an extension.
  - Move termainls into a separate jupyter server extension repository.
  - This extension can, then, be disabled. There are certainly contexts where people don't want to allow terminals (the authorization conversations below is one instance).
  - Make Jupyter Server require this as a dependency temporarily.
  - In a future release, we can drop it as a dependency.
Min is picking up the Authorization layer work.
- Security concerns for users
  - Many people deploying Server don't fully understand all the pieces.
  - We don't want to offer a system that could cause deployers to unknowingly deploy an insecure system.
  - We need to be sure to document what "execute" permissions means.
  - Once you have arbitrary code execution, you can change permissions on the server.
  - "execute" on one document essentially means you have "rwx" on all documents (i.e. the whole server) in a server with local kernels.
  - Ensure that users of Jupyter Server understand the risk of enabling "execute".
- Add the ability to set permissions on individual documents.

Zsailer commented 2 years ago

December 16th, 2021

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Rahul Goyal	AWS	@rahul26goyal
Vidar Fauske	JP Morgan Chase	@vidartf
Carlos Herrero	QuantStack	@hbcarlos
Craig Citro	Google	@craigcitro
Mariko Wakabayashi	OpenZeppelin	@mwakaba2

Agenda + Minutes

Insights from the week https://github.com/jupyter-server/jupyter_server/pulse
Follow up to #636
- Add a downstream test to our CI to check other kernels (e.g. R kernel).
No meetings the next two weeks.
Jupyter Server Team compass documentation and membership
- https://github.com/jupyter-server/team-compass/pull/14

Zsailer commented 2 years ago

It's time to close out this thread for a new year. Congratulations, all, on a fantastic year around the Jupyter-Server project! 🎉

Thank you to everyone who participated in these meetings. I'm grateful for this awesome community and look forward to another great year ahead. ❤️

jupyter-server / team-compass

Jupyter Server Notes 2021 #4

Hello everyone,

Meeting Notes

January 7th, 2021

Attendees

Agenda + Minutes

January 14th, 2021

Attendees

Agenda + Minutes

January 21st, 2021

Attendees

Agenda + Minutes

January 28th, 2021

Attendees

Agenda + Minutes

February 4th, 2021

Attendees

Agenda + Minutes

February 11th, 2021

Attendees

Agenda + Minutes

February 18th, 2021

Attendees

Agenda + Minutes

February 25th, 2021

Attendees

Agenda + Minutes

March 4th, 2021

Attendees

Agenda + Minutes

March 11th, 2021

Attendees

Agenda + Minutes

March 18th, 2021

Attendess

Agenda + Minutes

April 1st, 2021

Attendess

Agenda + Minutes

April 8th, 2021

Attendess

Agenda + Minutes

April 15th, 2021

Attendees

Agenda + Minutes

April 22nd, 2021

Attendees

Agenda + Minutes

April 29th, 2021

Attendees

Agenda + Minutes

May 6th, 2021

Attendees

Agenda + Minutes

Jupyter Server Weekly Meeting

May 13th, 2021

Attendees

Agenda + Minutes

May 27th, 2021

Attendees

Agenda + Minutes

June 3rd, 2021

Attendees

Agenda + Minutes

June 17th, 2021

Attendees

Agenda + Minutes

June 24th, 2021

Attendees

Agenda + Minutes

July 1st, 2021

Attendees

Agenda + Minutes

July 15th, 2021

Attendees

Agenda + Minutes

July 22th, 2021

Attendees

Agenda + Minutes