Meeting Notes 2022 - Githubissues

Zsailer commented 2 years ago

Hello everyone,

Welcome to the Jupyter Server Team!

You can find us in our Gitter chat room or visit us at our weekly meetings!

We meet on Thursdays at 8:00am, Pacific Standard Time on Jupyter's Zoom Channel. You can add yourself to the weekly agenda here. Everyone is welcome!

Let's avoid using this thread for discussion. If you'd like to discuss something in the minutes, open a separate issue and reference this thread.

Meeting Notes

Zsailer commented 2 years ago

January 6th, 2022

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Vidar T Fauske	JP Morgan Chase	@vidartf
David Brochart	QuantStack	@davidbrochart
Eric Charles	Datalayer	@echarles
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin
Mariko Wakabayashi	OpenZeppelin	@mwakaba2
Sylvain Corlay	QuantStack	@SylvainCorlay
Rahul Goyal	AWS	@rahul26goyal
Frederic Collonval	QuantStack	@fcollonval

Agenda + Minutes

Terminals as a server extension (initial work):
- New repo: https://github.com/jupyter-server/jupyter_server_terminals
- https://github.com/jupyter-server/jupyter_server/pull/651
  - Remove terminals from Jupyter Server
  - Requires jupyter_server_terminals
  - Maintains backwards compatibility, for now.
  - Next major release will not require terminals extension.
Jupyter Server Team
- https://github.com/jupyter-server/team-compass/pull/14
- An important note for how the Server team is being formed:
  - We (accidently, :sweat_smile:) side-stepped the official approach, defined by Jupyter Governance, to form a decision making body.
  - This process requires a current steering council member to seed (i.e. nominate an initial member). Together, the gradually growing team continues nominating new members.
  - Large, diverse, and inclusive teams is encouraged.
  - We, instead, self nominated ourselves, proposed by @Zsailer.
  - @afshin mentioned in the meeting that this is might be okay
    - we made a good-faith effort at nominating people
    - the current group that's actively participating is relatively small
    - we arrived at the same results anyways.
  - For other teams who might be observing this process, do not follow our process :laughing:
- Changes needed in the PR before merging:
  - Absolve the Team Lead position.
  - Define Software Steering Council Representative (we can vote on this person later).
  - Eliminate team colors.
    - General consensus is that we shouldn't distinguish between red and blue.
  - We just have an "active" team member (voting ability) and a "not currently active" team members (no voting ability).
Filtering request using OpenAPI spec: https://github.com/jupyter-server/jupyter_server/pull/645
- All handlers are added to the server, but requests from the user can be blocked/allowed by a defined OpenAPI Spec.
- There is some overlap with the current work happening around "Authorization" in Jupyter Server
- Is there a way to merge these two efforts so there aren't two ways to filter requests?
- Not immediate urgency to merge this work; we'll take some time to compare with authorization work.
Kernel Parameterization needs a reboot.
- Form a working group for this work?
  - Meet semi-regularly to discuss this topic.
  - Open a thread on jupyter-server/team-compass repo to begin nucleating around this topic.
- <= (echarles) Plan to demo https://github.com/deshaw/ksmm at next JupyterLab meeting (can be used a plan B for parametrized kernels)

Zsailer commented 2 years ago

January 13th, 2022

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Vidar T Fauske	JP Morgan Chase	@vidartf
David Brochart	QuantStack	@davidbrochart
Sylvain Corlay	QuantStack	@SylvainCorlay
A. T. Darian	Two Sigma	@afshin
Kevin Bates	IBM	@kevin-bates
Max Klein	JP Morgan Chase	@telamonian
Mariko Wakabayashi	OpenZeppelin	@mwakaba2
Eric Charles	Datalayer	@echarles

Agenda + Minutes

Protocol Alignment Proposal
- See the PR for full description. Kudos to David for a clear, well-written proposal. :tada:
- We need to support both protocols (old and new). We can change the default in a major release sometime in the future.
- Question from the group: do we use a versioned endpoint or query parameter to select the protocol version?
- It might be possible to use websocket protocols fields to select the protocol: https://developer.mozilla.org/en-US/docs/Web/API/WebSocket/WebSocket https://html.spec.whatwg.org/multipage/web-sockets.html#the-websocket-interface, there is a question if this could be made backwards-compatible though.
One 0MQ channel per kernel
Jupyter Server 1.13.2 released this week.
- https://github.com/jupyter-server/jupyter_server/releases/tag/v1.13.2
Finalizing the Jupyter Server Team
- Outstanding comments/questions.
- https://github.com/jupyter-server/team-compass/pull/14
Pending Kernels update
- Starting, Shutting down, restarting, and interrupting are all pending states.
- PR in Jupyter Client https://github.com/jupyter/jupyter_client/pull/732
- Updated Jupyter Server Tests: https://github.com/jupyter-server/jupyter_server/pull/654
Questions about jupyter_server/jupyter_client
- In the case of ZMQ over TCP, how are the sockets connected?
- What part of the code consumes the connection file?
  - In local model, these are managed and consumed by local kernel provisioners
KSMM (Kernelspec Manager) Demo
- https://github.com/deshaw/ksmm
- Demo from Eric Charles
- Parameterize Kernels by generating static kernelspecs using a templating engine
- Combine with remote_ikernel to generate remote kernels from these specs.

Zsailer commented 2 years ago

January 20th, 2022

Attendees

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin
Steven Silvester	MongoDB	@blink1073
David Brochart	QuantStack	@davidbrochart
Sylvain Corlay	QuantStack	@SylvainCorlay
Vidar T Fauske	JP Morgan Chase	@vidartf
Craig Citro	Google	@craigcitro

Agenda + Minutes

Windows tests are failing from pending kernels work
- Really flaky on Windows.
- Might (temporarily) skip these tests on Windows to prevent other PRs from failing.
- Will open an issue to track this.
7.1.0: this version makes JupyterLab very slow
- Windows-only and unique to user (so far), possible b/c issue with < 1.12 server
- Conda defaults channel version-lag - what updates defaults? This channel is periodically updated - so version-lags are expected.
- See also: https://github.com/jupyterlab/jupyterlab/issues/11811
- Actions: Vidar is going to spend some time attempting to reproduce the issue and narrow down the server versions. The current hypothesis is that this may be related to the "nudge" logic that was recently extended in 1.13 to utilize two channels. Once we have more information, we'll move/wrap-up the issues.
Team Compass is Live!
Adding SC members who requested to be on the Jupyter Server decision-making body if they aren't already there: @ellisonbg, @ivanov, @damianavila
Software Steering Council Representative
- An issue will be opened asking for nominations, after which a vote will be called

Zsailer commented 2 years ago

January 27th, 2022

Name	affiliation	username
Jason Weill	AWS	@jweill-aws
Zach Sailer	Apple	@Zsailer
Damián Avila	2i2c & MADS	@damianavila
Vidar T Fauske	JP Morgan Chase	@vidartf
Steven Silvester	MongoDB	@blink1073
Kevin Bates	IBM	@kevin-bates
Eric Charles	Datalayer	@echarles
Piyush Jain	AWS	@3coins
Akshay Chitneni	Apple	@akshaychitneni
Craig Citro	Google	@craigcitro
Jason Grout	Databricks	@jasongrout

Agenda + Minutes

SSC Representative
- Nominations open for 1 more day
- Vote next week.
Any feedback?
- Inject session identifier into environment variable. https://github.com/jupyter-server/jupyter_server/pull/679 - See https://github.com/jupyter-server/jupyter_server/pull/679#issuecomment-1023412126 for a summary of the discussion
- Drop nbclassic/notebook? https://github.com/jupyterlab/jupyterlab/pull/11894#issuecomment-1023318113
  - There's been discussion happening in the notebook meetings around this topic.
  - The current plan is to set nbclassic's requirement on lab to jupyterlab<4. JupyterLab 4 and nbclassic cannot run in the same environment.
- Terminado 0.13 is out with a new hook that helps get us last_activity. Corresponding PR in jupyter_server_terminals
Moving Gateway repos (Jupyter Kernel Gateway, Jupyter Enterprise Gateway) into jupyter-server organization?
- Kevin will open an issue on the team-compass (https://github.com/jupyter-server/team-compass/issues/20)
Discussion: how to onboard new contributors
- "Good first issue" label might deter some contributions. If you've already contributed once, you might feel guilty picking up another one.
- Let's not treat "good first issues" as so precious.
- Often times, someone "claims" a good first issue but never finish. This prevents others from picking up that issue.
- Host a weekly hack-a-thon/office hour meeting regularly?
  - Interested potential contributors can join and get help with contributing to Jupyter Server
  - Jupyter Server Team members help these contributors get their development environment setup, discuss areas where contributions are needed, teach them about the codebase, etc.
  - If no new contributors show up, Jupyter team members spend the time pair programming on some new software feature in Jupyter Server.

Zsailer commented 2 years ago

February 3rd, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Luciano Resende	Apple	@lresende
Jason Weill	AWS	@jweill-aws
Steven Silvester	MongoDB	@blink1073
Eric Charles	Datalayer	@echarles
Rahul Goyal	AWS	@rahul26goyal
Damián Avila	2i2c & MADS	@damianavila
Akshay Chitneni	Apple	@akshaychitneni
Paul Ivanov	Noteable	@ivanov
Craig Citro	Google	@craigcitro
A. T. Darian	Two Sigma	@afshin
Jessica Xu	@jess-x

Agenda + Minutes

Authorization PR ready for final review
- Final review between now and next Tuesday
- If nothing major shows up in the review, we'll plan to merge on Tuesday! (Thank you, Min)!
Test Flakiness
- Possible conflict between pytest-tornasync, pytest-timeout, and asyncio.
Moving Gateways to Server org
- EG Update:
  - 2.6.0 release is out
  - Major doc re-org coming, along with 3.0 (Spark 3 support)
  - 4.0 will be based on kernel provisioners
Discussion of starting experimental repos within jupyter-server Github org
- Ok to use org as umbrella for experiments (instead of personal accounts)
- communicate explicitly in the README that it is experimental
Auto pre-commit - stick with the default GitHub Access Token for now?
Jupyter Server Office/Hacking Hour.
- Thoughts on this idea?
- If we like it, how often?
- Naming this hour session matters.
  - Calling it "office" hours may lead people to come with specific questions about their deployments.
  - Calling it "hacking hour" may lead to people doing a bunch of exploratory work
  - Calling it a "work party" may be more explicit in capturing the intent
SSC Representative
- Open an issue to vote.

Zsailer commented 2 years ago

February 10th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Max Klein	JPM	@telamonian
Kevin Bates	IBM	@kevin-bates
Jason Weill	AWS	@jweill-aws
A. T. Darian	Two Sigma	@afshin
Mariko W.	OpenZeppelin	@mwakaba2
Rahul G	AWS	@rahul26goyal
Eric Charles	Datalayer	@echarles
Piyush Jain	AWS	@3coins
Craig Citro	Google	@craigcitro

Agenda + Minutes

jupyter kernel init sequence diagram
- https://github.com/jupyter-server/enterprise_gateway/pull/1039
how to use pending kernels
- and with a gateway?
  - EG can't support pending kernels until it moves to provisioners (it's currently incompatible with juptyer_client 7.x)
  - KG (Kernel Gateway) could support pending kernels if we moved it to jupyter_server (its based on notebook IIRC).
PR for making contents hook support multiple hooks: https://github.com/jupyter-server/jupyter_server/pull/696/files
- Triggered by this discussion: https://github.com/jupyterlab/jupyterlab/pull/11599#discussion_r801963771
- Should probably have different ways to add/remove hooks as needed
- Order is probably important
- Uniqueness should probably be enforced in case the validator triggers multiple times?
- Maybe expose a new trait exposing the collection of hooks for manipulation.
Google Group for decision-making body
Question: Has there been any work in the past or on-going to improve the synchronization (autosave) of notebook to server.
- maybe https://github.com/jupyterlab/jupyterlab/issues/5787 ?
Releasing rules
- Open an issue in team compass to discuss rules around cutting new releases of software in the Jupyter Server org.

Zsailer commented 2 years ago

February 17th, 2022

Name	affiliation	username
Jason Weill	AWS	@jweill-aws
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
A. T. Darian	Two Sigma	@afshin
Rahul Goyal	AWS	@rahul26goyal
Mariko W	OpenZeppelin	@mwakaba2
Piyush Jain	AWS	@3coins
Craig Citro	Google	@craigcitro

Agenda + Minutes

Call for review: Identity API
- 4 design decision questions to review
Dependency Handling / upper capping dependencies
- https://github.com/jupyter-server/jupyter_server/issues/699
- We have two dependencies with caps
Call for review: Register pre/post save hooks, call them sequentially
Discussion around Kernel Specification Manager
- Provides a way to generate new kernel specs.
- Schemas from the server inform the client what parts/parameters of the kernel spec are configurable.
- In the UI, user can set the value of for each parameter.
- On save, a new kernel spec is created on the server that will be used to launch kernels with the given parameter values.
- VS https://github.com/jupyter/enhancement-proposals/pull/87
Our first "Contributing" hour is happening right after this call!

Zsailer commented 2 years ago

February 24th, 2022

Name	affiliation	username
Jason Weill	AWS	@jweill-aws
Kevin Bates	IBM	@kevin-bates
Zach Sailer	Apple	@Zsailer
Eric Charles	Datalayer	@echarles
Vidar T Fauske	JP Morgan Chase	@vidartf
Rahul Goyal	AWS	@rahul26goyal
Akshay Chitneni	Apple	@akshaychitneni
A. T. Darian	Two Sigma	@afshin
Andrey Velichkevich	Apple	@andreyvelich
Piyush Jain	AWS	@3coins

Agenda + Minutes

Making sense of the new world: Notebook, RetroLab, JupyterLab, NBClassic, Notebook Shim, and Jupyter Server
- Notebook v6 —> Tornado web server and legacy Notebook Javascript
- Notebook v7 —> JupyterLab components with classic Notebook UX (used to be RetroLab) + Jupyter Server
- JupyterLab —> JupyterLab UX + Jupyter Server
- NBClassic used to have 2 tasks
  - Adds classic Notebook JS and endpoints to Jupyter Server
  - ~~Handle shimming traits and config~~ (moved to notebook_shim)
- notebook_shim
  - Handle shimming traits and config
- What options will users have to run "side-by-side"?
  - Notebook v6, NBClassic, JupyterLab
  - Notebook v7 + JupyterLab
Current state of custom messages on the control channel?
- https://github.com/jupyterlab/jupyterlab/pull/11285
- Get resource usage from ipykernel: https://github.com/ipython/ipykernel/pull/805
- Can we relax the typing in JupyterLab to allow custom messages to the control channel?
  - This shouldn't require JEP.
  - Discussion on that can happen in https://github.com/jupyterlab/jupyterlab/issues/12117
- If it's just resource usage, perhaps provisioners are a better option. This was one of the motivations for provisioners.
- Zach's motivation is to push this one step further than resource usage metrics...
  - Can we get arbitrary "telemetry" events to flow from kernel to frontend client.
  - Open Telemetry could enable us to trace an event from the deepest layer of the stack back to the client.
  - The control channel looks like the best place to flow messages from the kernel to any kernel client.
  - Open Telemetry might be a standard we can add to Jupyter, but would require a JEP.
"Contributing" Hour right after this meeting!
- Introduced traitlets
- Introduced server extensions
- cover Jupyter Server unit tests next week.

jess-x commented 2 years ago

March 3, 2022

Name	affiliation	username
Jason Weill	AWS	@jweill-aws
Vidar T Fauske	JP Morgan Chase	@vidartf
Zach Sailer	Apple	@Zsailer
Steven Silvester	MongoDB	@blink1073
Rahul Goyal	AWS	@rahul26goyal
Kevin Bates	IBM	@kevin-bates
Kiersten Stokes	IBM	@kiersten-stokes
Akshay Chitneni	Apple	@akshaychitneni
A. T. Darian	Two Sigma	@afshin
Mariko W	OpenZeppelin	@mwakaba2
Andrey Velichkevich	Apple	@andreyvelich

Agenda + Minutes

Issue #312 - Improving Network Performance opened September 2020
- In looking at things, @goanpeca found that the server is validating notebooks twice - potential 50% savings per notebook!
  - Nbformat PR #249 adds the ability to capture validation errors]
  - TODO: On next nbformat release, update server to add the additional parameter, remove second validation, and bump dependency floor on nbformat
  - TODO: We should also enable the ability to leverage fastjsonschema that @goanpeca added to nbformat, exposed via env.
    - Should this be configurable, or fastjsonschema is auto-detected and used unconditionally? (leaning toward configurable in case issues arise or other options appear)
      - If configurable, should it expose (and track) available options or a boolean? (leaning toward enum, default = jsonschema)
- We should probably talk about static asset management posted in the issue as well.
Maintenence-related cleanups
- Terminal culling works again on Windows!
- Testing against min dependencies and pre-releases
- Removed pinning up upper version deps
- Working on cleaning up jupyter_client as well so we can handle warnings as errors as well
- Flit-based backend in jupyter-packaging coming soon
pre/post_save_hook PR: https://github.com/jupyter-server/jupyter_server/pull/696
- Urgent, because of issues in JupyterLab
- Not ready to merge in it’s current state.
- Current state/question: should there be a way to order these post_save_hooks?
- Thoughts:
  - The single value trait, post_save_hook, should be deprecated
  - Replace with a List trait, post_save_hooks, where order matters.
  - What happens if the same post_save_hook is given by an extension and by the user configuration? It would add the hook twice, which probably isn’t intended.
  - Need to put more thought into this.
Identity API /api/me
- This will likely require a new major version. It may be about time to do this.
- Adding this API is one of the critical building blocks for RTC
Websocket subprotocol bug: https://github.com/jupyter-server/jupyter_server/pull/706
- Causing issues with pending kernels
- Thank you, David, for quickly fixing!
“Contributing” hour right after this!

jess-x commented 2 years ago

March 10, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Vidar Tonaas Fauske	JP Morgan Chase	@vidartf
A. T. Darian	Two Sigma	@afshin
Kiersten Stokes	IBM	@kiersten-stokes
Rahul Goyal	AWS	@rahul26goyal
Jason Weill	AWS	@jweill-aws
Steven Silvester	MongoDB	@blink1073
Carlos Herrero	QuantStack	@hbcarlos
Akshay Chitneni	Apple	@akshaychitneni
Andrey Velichkevich	Apple	@andreyvelich
Craig Citro	Google	@craigcitro

Agenda + Minutes

Darian: Governance questions
- https://github.com/jupyterlab/team-compass/pull/139
- "Council"
  - Change the current usage of "team" to "council"
- Voting
  - Change our voting process around electing new team members from public to a private mailing list.
  - Anytime we're voting on people, we should do this privately.
Authorization PR introduced a backwards incompatible change
- Using jupyter_server handlers, but no its webapp causes issues if the custom webapp doesn't provide an authorizer.
- Make Authorizers singletons?
- If the tornado web app doesn't have an authorizer in the settings, call Authorizer.instance(...)
Work in progress: https://github.com/jupyter/jupyter_client/pull/751 to clean up pending kernels and asyncio usage in jupyter_client and jupyter_server.
- Drop the .ready (a Future) property in the KernelManager.
- Python's core documentation recommends not using Futures directly.
- Use public methods to synchronously check if a pending kernel is ready.
- Drop nested looks, i.e. nested_asyncio.
absolute vs resolve https://github.com/jupyter-server/jupyter_server/pull/712: some interesting discussion that might be relevant in https://discuss.python.org/t/pathlib-absolute-vs-resolve/2573/2 and https://bugs.python.org/issue39090
"Contributing" hour right after this!

Zsailer commented 2 years ago

March 17th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Jason Weill	AWS	@jweill-aws
Kiersten Stokes	IBM	@kiersten-stokes
A. T. Darian	Two Sigma	@afshin
Jessica Xu	Jupyter community	@jess-x
Rahul	AWS	@rahul26goyal
Andrey Velichkevich	Apple	@andreyvelich
Piyush Jain	AWS	@3coins

Agenda + Minutes

Add Carol to Jupyter Server Council :tada:
Governance fun times
- Zach and Darian are working on a team-compass template for other Jupyter subprojects.
- The plan is to dogfood the processes described in the template here in Jupyter Server.
Cal Poly's Notifications extension
- Contains a server and jupyterlab extension
- Should some of the server-side endpoints land in core server?
- Needs a websocket for broadcasting events to the frontend
- Needs a REST API for creating and fetching notifications.
- Jupyter Eventlogging/telemetry plans to implement plumbing that notifications can use.
Telemetry/EventLogging
- Should we track https://github.com/jupyter-server/jupyter_server/pull/501
- What is the future of telemetry within JupyterLab? Are there any future/current proposals?
- FYI: Working on upgrading jupyterlab-telemetry to work with JL3. https://github.com/jupyterlab/jupyterlab-telemetry/issues/23
- Relevant links for Eventlogging in Jupyter
  - JEP: https://github.com/jupyter/enhancement-proposals/pull/41
  - Core Jupyter Telemetry Python Library: https://github.com/jupyter/telemetry
  - Draft PR adding an eventlog to Jupyter Server: https://github.com/jupyter-server/jupyter_server/pull/364
Ready for review? https://github.com/jupyter-server/jupyter_server/pull/679
"Contributing" hour right after this!
- Discussed the relationship between JupyterHub and Jupyter Server
- Wrote a basic Jupyter Server extension from scratch

Zsailer commented 2 years ago

March 24th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
A. T. Darian	Two Sigma	@afshin
Steven Silvester	MongoDB	@blink1073
Marc Udoff		@mlucool
Kiersten Stokes	IBM	@kiersten-stokes
Andrey Velichkevich	Apple	@andreyvelich
Kevin Bates	IBM	@kevin-bates
Frederic Collonval	QuantStack	@fcollonval
Piyush Jain	AWS	@3coins
Jessica Xu	Jupyter	@jess-x
Rahul Goyal	AWS	@rahul26goyal

Agenda + Minutes

Zach
- Add hook to observe pending sessions
- Kernel and session rehydration and syncing
Marc
- nginx + static assets
- Marc plans to work on a PR to Jupyter Server
Steve
- We can now use @meeseeksdev please run pre-commit to run pre-commit on PRs. I opened an issue on JupyterLab about switching from husky to pre-commit.
- Prototyping the wrapping of an asynchronous class as part of my #dayjob work, will be applicable to jupyter_client
EG issue
- FD add twice leak EG#1051
Frederic:
- Need help to analyze a asyncio bug breaking JLab at start up.
Piyush
- jupyterlab-telemetry v1.0.3 released to NPM and PyPi, this is compatible with JL3 now.
- Write the telemetry events to a separate log #31
"Contributing" hour right after this!
- Explored what "sessions" are in Jupyter Server
- Demonstrated how sessions work in JupyterLab
- Discussed the Synchronizer proposed above and how it relates to sessions.

mwakaba2 commented 2 years ago

March 31st, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
David Brochart	QuantStack	@davidbrochart
Jason Weill	AWS	@jweill-aws
Mariko W.	OpenZeppelin	@mwakaba2
Kevin Bates	IBM	@kevin-bates
Andrey Velichkevich	Apple	@andreyvelich
Kiersten Stokes	IBM	@kiersten-stokes
Piyush Jain	AWS	@3coins

Agenda + Minutes

Released v1.16.0 this week: https://github.com/jupyter-server/jupyter_server/releases/tag/v1.16.0
Bug Report: XSRF cookie not always set on first request
- look into change history in jupyter/notebook
This issue was mentioned in JupyterLab meeting and last week's meeting. https://github.com/jupyterlab/jupyterlab/issues/11934
- We're hoping that removing the nest_asyncio monkeypatching in jupyter_client will fix this issue.
- https://github.com/jupyter/jupyter_client/issues/763
- https://github.com/jupyter/jupyter_client/issues/755
Should we begin a Jupyter Server 2.0 branch
- identity API PR is introducing backwards incompatibility
  - Plan to remove the LoginHandler and replaces with IdentityProvider in future PR
  - While we can add shims to address the incompatibility, it probably still warrants a major release
- We should have some rules around the duration of major releases.
  - What is the minimum time we should have between major releases?
  - Let's manage the expectations of people watching this repo.
- Fixed calendar cycle of major releases?
- Mention this major release to the JupyterLab team.
  - Could potentially have lab check for idenitiy API presence at runtime and not make a install time requirement on server 2.0.
"Contributing" hour right after this!

Zsailer commented 2 years ago

April 7th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Steven Silvester	MongoDB	@blink1073
David Brochart	QuantStack	@davidbrochart
A. T. Darian	QuantStack	@afshin
Kevin Bates	IBM	@kevin-bates
Andrey Velichkevich	Apple	@andreyvelich
Akshay Chitneni	Apple	@akshaychitneni
Piyush Jain	AWS	@3coins
Kiersten Stokes	IBM	@kiersten-stokes
Rahul Goyal	AWS	@rahul26goyal

Agenda + Minutes

Darian's affiliation
Big "shout out" to Steve (@blink1073) for updating the Jupyter repos (including those outside the Jupyter Server org) to have a consistent CI experience. This is really nice from a "maintainer-familiarity" perspective and much appreciated! Thank you Steve! :tada: :heart:
Events: https://github.com/jupyter-server/jupyter_server/issues/780
- Proposing to add 4 REST endpoints and Websocket to core Jupyter Server
- Darian's goal is to get some form of this in by early summer—even if it's a simple/lightweight version.
Jupyter Server 2.0 Discussion
- New features for 2.0
  - Identity
    - Is this needed for JupyterLab 4.0, i.e. before June? No.
  - Events
  - Remove Terminals
- Do we need JupyterLab 4 to require Jupyter Server 2.0?
- Next Steps:
  - [x] Create a milestone
  - [ ] Create 1.x branch and make main the 2.x.
  - [ ] Merge Identity PR
  - [ ] Cut a 2.0 pre-release
Released [EXPERIMENTAL] Synchronizer extension
- Requires an addition in the GatewayClient API in (core) Jupyter Server—a new method called list_kernels
- Can we use the list_kernels from the MultiKernelManager.
Extensions configuring other extensions?
- Ran out of time to discuss this thoroughly. We'll discuss further next week.
- Do we need a dependency injection system?
- Some previous discussion: https://github.com/jupyter-server/jupyter_server/issues/536
"Contributing" hour right after this!
- - Adding a websocket handler as a server extension

fcollonval commented 2 years ago

Is needed for JupyterLab 4.0, i.e. before June? No.

I disagree on that point. Identity is needed for RTC and we certainly hope to get a first decent RTC experience in 4.0.

jess-x commented 2 years ago

April 14th, 2022

Name	affiliation	username
Frederic Collonval	QuantStack	@fcollonval
Afshin T. Darian	QuantStack	@afshin
Jason Weill	AWS	@jweill-aws
Andrey Velichkevich	Apple	@andreyvelich
Kevin Bates	IBM	@kevin-bates
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Piyush Jain	AWS	@3coins
Akshay Chitneni	Apple	@akshaychitneni
Craig Citro	Google	@craigcitro

Agenda + Minutes

Notification/Telemetry endpoints should be available prior to Lab 4.0 (Server 2.0)
main branch of jupyter_server needs to target 2.x before we merge backward-incompatible things like identity API
(Andrey) Share few Jupyter Server diagrams
Demo of Jupyter Server Synchronizer.
Contributing Hour right after this!

jess-x commented 2 years ago

April 21st, 2022

Name	affiliation	username
Steven Silvester	MongoDB	@blink1073
Vidar T Fauske	JP Morgan Chase	@vidartf
Jason Weill	AWS	@jweill-aws
Kevin Bates	IBM	@kevin-bates
Kiersten Stokes	IBM	@kiersten-stokes
Rahul Goyal	AWS	@rahul26goyal
A. T. Darian	QuantStack	@afshin
Andrey Velichkevich	Apple	@andreyvelich
Jessica Xu

Agenda + Minutes

Follow up from last week on ServerApp Cleanup
WIP Kernel Status Cleanup Proposal
v2 vs 1.x branch progress: https://github.com/jupyter-server/jupyter_server/issues/789
Use the flit backend directly instead of Jupyter Server? Would mean an explicit build step before making the sdist or wheel. If you don't run the npm build step the default pages just aren't styled.
Where was the "pop empty from deque" issue?: https://github.com/jupyterlab/jupyterlab/issues/11934
"Contributing" hour right after this!
- - Jupyter-server extension cookiecutter

afshin commented 2 years ago

April 28th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Steven Silvester	MongoDB	@blink1073
Rahul Goyal	AWS	@rahul26goyal
Brian Granger	AWS/CalPoly	@ellisonbg
Kiersten Stokes	IBM	@kiersten-stokes
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Kevin Bates	IBM	@kevin-bates
Kien Dang	NUS	@kiendang
Adam Patterson	UM	adpatter
Andrey Velichkevich	Apple	@andreyvelich
Craig Citro	Google	@craigcitro

Agenda + Minutes

Welcome, Rahul, to the Jupyter Server Team :tada:
Jupyter Event System and Telemetry
- Introductions
- Press Release
- Design Document
- Jupyter Enhancement Proposal
- Jupyter Telemetry library
- Event logging in JupyterHub: https://github.com/jupyterhub/jupyterhub/pull/2698
- Next steps:
  - [x] Configurable event logger with schema validation
  - [ ] EventBus: a websocket endpoint for broadcasting/subscribing to events
  - [ ] Endpoint for posting events
  - [ ] JupyterLab extension for listening/consuming events
  - [ ] PII considerations
  - [ ] Add core schemas/event types.
- cf. https://github.com/jupyter-server/jupyter_server/issues/780
- Discussion/Comments
  - Telemetry, EventLogging, Events, Notifications have different needs driving their development, so they will likely require different systems.
    - Telemetry must have an emphasis on handling PII properly
    - Notifications has subtle logic in a multi-user context that might not have the same subtlty in Telemetry.
  - EventBus is lowest common denominator between Telemetry, EventLogging, Notifications. Can we get this into Jupyter Server 2.0?
    - i.e. endpoint for creating/fetching/broadcasting events.
PyCON Packaging Summit tomorrow - Steve will be presenting about External Data - a replacement for the now deprecated data_files.
"Contributing" hour right after this!

Zsailer commented 2 years ago

May 5th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Steven Silvester	MongoDB	@blink1073
Piyush Jain	AWS	@3coins
Vidar T Fauske	JP Morgan Chase	@vidartf
Jason Weill	AWS	@jweill-aws
Andrii Ieroshenko	Cal Poly Jupyter	@andrii-i
Afshin T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Kevin Bates	IBM	@kevin-bates
Kiersten Stokes	IBM	@kiersten-stokes
Adam Patterson	US	adpatter
Matt Henderson	LBNL	@mlhenderson
Rahul Goyal	AWS	@rahul26goyal
Andrey Velichkevich	Apple	@andreyvelich

Agenda + Minutes

Zach
- Basic Event Bus
- Enable GatewayClient managers to be configurable: https://github.com/jupyter-server/jupyter_server/pull/834
- SciPy 2022, anyone else going? (Steve is)
Steve
- v2.0.0a0 release
- Flit backend - one potential gotcha with prereleases
- External Data proposal - Need to write a PEP
Piyush
- Discuss tasks for Event Notification System, intent would be to open PRs for each of the numbered items.
  1. Basic Event Bus - #820
    - A central event bus to relay events
    - /api/events/subscribe - Websocket for subscribing to events
    - A default handler for consuming events
  2. Rest API Endpoints
    - POST /api/events - Rest api to create new events
    - Do we need API for listing topics/schemas? May be add this is a separate PR.
      - /api/events/schemas?
  3. JupyterLab 4 Event Client (jupyterlab-events)
    - Rename jupyterlab-telemetry repo to jupyterlab-events
    - Remove server endpoints, any redundant server code
    - Update client handlers to use the rest api endpoints
    - Add websocket handler to enable subscribtion to events
  4. Default events in server
    - Add default events e.g., content handler, kernel events in jupyter server
  5. JupyterLab 4 Updates
    - Add jupyterlab-events to JupyterLab
    - Subscribe to default events
  6. Event Notification UI (JupyterLab)
    - UI updates for event notification
  7. Event buffer (Move to #2 or #3)
    - A queue/buffer to store event messages if not delivered by the websocket
  8. Jupyter Notebook 7 Updates
    - Add jupyterlab-events
    - Subscribe to default events
    - Can we reuse event notification UI from #6?
  9. Telemetry?
David
- Move jupyter_ydoc to https://github.com/jupyter-server ? I opened an issue on the team-compass.

Zsailer commented 2 years ago

May 12, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Piyush Jain	AWS	@3coins
David Qiu	AWS	@dlqqq
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Andrii Ieroshenko	Cal Poly Jupyter	@andrii-i
Kiersten Stokes	IBM	@kiersten-stokes
Kevin Bates	IBM	@kevin-bates
Jessica Xu	n/a	@jess-x
Kien Dang	NUS	@kiendang
Andrey Velichkevich	Apple	@andreyvelich
Rahul Goyal	AWS	@rahul26goyal
Carlos Herrero	QuantStack	@hbcarlos
Rosio Reyes	Anaconda	@RRosio

Agenda + Minutes

Events Talk™ (notifications edition)
- First pass: sqlite + labextension
- Second pass: Yjs? + labextension
- Third pass: sqlite + labextension
- Questions mentioned during the meeting:
  - Are we broadcasting all events?
  - Are we doing some topic/subtopic matching?
  - How does identity play into routing events?
  - How do we handle multiple clients (same identity)?
- Create jupyter_events package and move most of telemetry into that package.
- Event Bus
  - PR ready for final review
- Filtering subscriptions from the client
  - e.g. /api/events/subscribe/notifications
    - Creates WebsocketHandler with a filter that accepts only "notification" events.
- Let's create a HackMD as a working doc for the Event System:
  - https://hackmd.io/q4Rkq2BaS1SIXvyzt8j1yA

Zsailer commented 2 years ago

May 19, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Andrii Ieroshenko	Cal Poly Jupyter	@andrii-i
Jason Weill	AWS	@jweill-aws
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Vidar T Fauske	JP Morgan Chase	@vidartf
Kien Dang	NUS	@kiendang
Steven Silvester	MongoDB	@blink1073
Piyush Jain	AWS	@3coins
Akshay Chitneni	Apple	@akshaychitneni
Rahul Goyal	AWS	@rahul26goyal

Agenda + Minutes

Zach
- Basic Event Bus PR ready for (hopefully final) review
- In current implementation, the /api/events/subscribe endpoint sends all events across the websocket.
- We discussed that we need to add a way to open a websocket that only streams a subset of events.
- Proposed plan:
  - Client hits /api/events/subscribe to get a websocket connection
  - The server responds with a READY or IDLE message to notify client that the connection is available to stream data.
  - The client sends a list of filters to apply (or empty list for all events)
Vidar
- pop from deque: https://github.com/jupyterlab/jupyterlab/issues/11934 Seems related to multiple(?) kernels starting up at the same time?
Steve (will be 15 min late)
- Using the hatch build plugin
- Python 3.11 - still blocked by upstream PyYAML
"Contributing" hour right after this!

Zsailer commented 2 years ago

May 26, 2022

Name	affiliation	username
Vidar T Fauske	JP Morgan Chase	@vidartf
Steven Silvester	MongoDB	@blink1073
Rick Wagner	UCSD	@rpwagner
Kiersten Stokes	IBM	@kiersten-stokes
Jason Weill	AWS	@jweill-aws
A. T. Darian	QuantStack	@afshin
Zach Sailer	Apple	@Zsailer
Rahul Goyal	AWS	@rahul26goyal
Andrey Velichkevich	Apple	@andreyvelich
Piyush Jain	AWS	@3coins
Rosio Reyes	Anaconda	@RRosio

Rick & Jason W
- Vulnerability handling process
Vidar
- I see AttributeError: 'AsyncIOLoopKernelManager' object has no attribute 'last_activity', which indicates a race condition at least for that property. That might also be a clue for the pop from deque? https://github.com/jupyterlab/jupyterlab/issues/11934
Steve (will be 15-20 min late)
- Update on hatch_jupyter_builder
- Zach will add himself as a maintainer of conda-forge
Darian
- Jupyter Kernels Subproject question (team-compass)
- Question about downstream development that relies on the main branch:
- notebook_shim
- jupyterlab_server
- jupyterlab
Piyush
- Binder doesn't work for serving api endpoints in a server extension. This might be due to binder not using the jupyter-server. How can we update this?
- Contributing hour questions:
  - How to setup resources e.g., a database for a extension application that is available to the API handlers?

Zsailer commented 2 years ago

June 2nd, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Steven Silvester	MongoDB	@blink1073
Kiersten Stokes	IBM	@kiersten-stokes
Kevin Bates	IBM	@kevin-bates
Rahul Goyal	AWS	@rahul26goyal
Piyush Jain	AWS	@3coins

Zach
- Created and released jupyter_events
  - Replace jupyter_telemetry with jupyter_events in Jupyter Server
  - Waiting for conda-forge recipe to be accepted: https://github.com/conda-forge/staged-recipes/pull/19123
  - Next step is to make jupyter_telemetry depend on jupyter_events.
Steve
- Update on hatch_jupyter_plugin progress
- Migration helpers are mostly done, needs tests
- notebook and extension-cookiecutter-ts are blocked right now due to limitations in JupyterLab's handling of setup.py and symlinks.
Kevin (EG update)
- Hope to get a 3.0 beta out soon (following merge of pending PRs)
- Open PRs: "Availability Modes", "Tenant ID" support, WebhookKernelSessionManager
Rahul
- Notebook Cell output messages are not available while the client connection from UI is lost due to various reasons. -- Scenario 1: User has started executing a notebook cell and closed the notebook while the execution is still happening and outputs are generated. -- Scenario 2: User has started excuting a notebook cell and due to network issue, UI has lost the connectivity to Jupyter server. -- Scenario 3: User had started executing a notebook cell, closed the notebook and opens again.
Could consider renaming event category to Redaction Policy
"Contributing" hour right after this!

kevin-bates commented 2 years ago

June 9nd, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Vidar T Fauske	JP Morgan Chase	@vidartf
Trung Le	QuantStack	@trungleduc
Piyush Jain	AWS	@3coins
Rahul G	AWS	@rahul26goyal
A. T. Darian	QuantStack	@afshin
Kevin Bates	IBM	@kevin-bates
Akshay Chitneni	Apple	@akshaychitneni
Andrey Velichkevich	Apple	@andreyvelich

Trung Le
- Move jupyterlab-lsp server extension to jupyter-server org ? https://github.com/jupyter-server/team-compass/issues/30
- Question: will this add friction to the development of this extension, since the server and frontend pieces will exist in two different places with two different teams managing these pieces?
Zach
- Jupyter Events, category filter. Is this a telemetry-specific feature?
- Jupyter Server Examples
  - Move examples to separate repo?
  - Examples are currently broken—maybe something we can do at contributing hour
"Contributing" hour right after this!
Rahul
- Not directly related to J_S, but how do we pass parameters from kernel launcher to a kernel magic library. I usually do this by setting an ENV variable but I think there must be other ways to pass this via some ipython args.
Piyush
- How to get the notebook dir path in a handler API? For example, if jupyter lab started with --notebook-dir=<dir>, how to resolve this in a handler?
  - serverapp.root_dir?
  - settings - server_root_dir

Zsailer commented 2 years ago

June 16th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Kevin Bates	IBM	@kevin-bates
Sylvain Corlay	QuantStack	@SylvainCorlay
David Brochart	QuantStack	@davidbrochart
Steven Silvester	MongoDB	@blink1073
Andrii Ieroshenko	Cal Poly Jupyter	@andrii-i
Vidar T Fauske	JP Morgan Chase	@vidartf
Rahul Goyal	AWS	@rahul26goyal
Kiersten Stokes	IBM	@kiersten-stokes
A. T. Darian	QuantStack	@afshin
Andrey Velichkevich	Apple	@andreyvelich

Sylvain Corlay Discussing move of Jupyter-LSP to jupyter-server organization
- Approved by vote during the meeting
Steve
- "Error on Warnings" Policy
  - Constrain to a CI job with opt-in
  - Ignore warning from pytest-tornasync for now
  - Track and attempt to patch/upstream a fix if possible by next fall when Python 3.12 comes out.
- Hatchling Update - notebook repo is converted, JupyterLab/extensions migration under way
David mentioned work going on in ipykernel
- Allows parallel execution in a single kernel
- E.g. have a long running cell and another cell visualizing the results of that cell.
- https://github.com/ipython/ipykernel/pull/955
Kernels Subproject
- Use the last 10-15 minutes of the Jupyter Server meetings for the Kernel subproject meeting?
No fix for https://github.com/jupyterlab/jupyterlab/issues/11934
Vidar and Darian are doing a contribution sprint this weekend, please add "Good first" issue labels if you know a good candidate!
"Contributing" hour right after this!

Zsailer commented 2 years ago

June 23rd, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Jason Weill	AWS	@jweill-aws
Vidar T Fauske	JP Morgan Chase	@vidartf
Kevin Bates	IBM	@kevin-bates
Kiersten Stokes	IBM	@kiersten-stokes
A. T. Darian	QuantStack	@afshin
Andrii Ieroshenko	Cal Poly Jupyter	@andrii-i
David Qiu	AWS	@dlqqq
Steven Silvester	MongoDB	@blink1073
David Brochart	QuantStack	@davidbrochart
Piyush Jain	AWS	@3coins
Andrey Velichkevich	Apple	@andreyvelich

Sylvain
- Content API change proposal (I need to go 1st because I have another call 15min later with NumFOCUS).
- Related issue: https://github.com/jupyter-server/jupyter_server/issues/884
- Remove (or maybe ignore) "notebook" type in Contents API
- Thus, remove the notebook JSON validation step server-side
- Removes nbformat as a dependency.
  - nbformat is used for "trusting" cells
  - nbformat is used to update notebook format versions on the fly
- This is a backwards incompatible change.
- We should aim to avoid breaking custom ContentsManager implementations. At very least, we should support old ContentsManagers for one deprecation/major release cycle.
Zach
- Memory leak in jsonschema that's affecting Jupyter Events/Telemetry
  - This was seen before: https://github.com/python-jsonschema/jsonschema/issues/237
- I'll be out of office next week.
- David Q: possibly related? https://github.com/python-jsonschema/jsonschema/issues/868
Vidar
- Started trying to repro pop from deque, but not able to reproduce it (yet).
Steve
- Released 1.18.0
- More progress on hatch migration script and cookiecutter update
David Qiu
- file ID service discussion
- https://github.com/jupyterlab/jupyterlab/issues/12614
- Biggest challenge is that out-of-band changes cause this service to get out of date/sync.
Darian
- Questions about events that arose in: @jupyterlab/services PR for events WebSocket and POST handlers
  1. Do we ship with a front-end schema for events or should JupyterLab ship with one (in jupyterlab_server? somewhere else?)
  2. Is record_event and the verb record what we should go with? The EventManager#record method is meant to echo the back-end's API

blink1073 commented 2 years ago

June 30th, 2022

Name	affiliation	username
Steven Silvester	MongoDB	@blink1073
Vidar T Fauske	JP Morgan Chase	@vidartf
A. T. Darian	QuantStack	@afshin
Kevin Bates	IBM	@kevin-bates
David Brochart	QuantStack	@davidbrochart

[Vidar] Server startup failure: https://github.com/jupyterlab/jupyterlab/issues/11934
- Some new information on the issue, but no real progress.
[Steve] Hatch migration status update
- Widget extension cookiecutter PR is done
- Updating the migration script to handle older APIs in jupyter_packaging
- Next up is the lab extension js cookiecutter
[David] Register new file type handlers:
- https://github.com/jupyter-server/jupyter_server/issues/884
- https://github.com/jupyter-server/jupyter_server/pull/895
- [Kevin] Gateway Client: Looking into adding the ability to renew token used to access Gateway Server
- Improves the use of deploying a Gateway in hosted environments

kevin-bates commented 2 years ago

July 7th, 2022

Name	affiliation	username
Steven Silvester	MongoDB	@blink1073
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Kiersten Stokes	IBM	@kiersten-stokes
Kevin Bates	IBM	@kevin-bates
Jason Weill	AWS	@jweill-aws
A. T. Darian	QuantStack	@afshin
Andrey Velichkevich	Apple	@andreyvelich
Rahul Goyal	AWS	@rahul26goyal

Agenda

Vidar
- #11934: When upgrading to tornado 6.2, we also see errors like this: https://github.com/erdewit/nest_asyncio/issues/22
- Better handling of async loops - https://github.com/jupyter/jupyter_client/issues/755
Zach
- SciPy next week.
  - Working on a Jupyter Server 2.0 release.
- Jupyter Events memory leak update.
  - Fixed in main branch, but not in latest release.
  - Caused by jsonschema (see gist).
  - From the docs: "If you know you have a valid schema already, especially if you intend to validate multiple instances with the same schema, you likely would prefer using the Validator.validate method directly on a specific validator (e.g. Draft7Validator.validate)."
  - We've switched to a custom validator that might fix this: https://github.com/jupyter/telemetry/blob/4c7bd069cc8432f92792376861b4746d7bbedb6a/jupyter_telemetry/eventlog.py#L229
- Merge: https://github.com/jupyter-server/jupyter_server/pull/825
Steve
- Hatch progress - cookiecutters updated, working on documentation
- Releaser bug with "next" and "patch" version handling
Further Discussions
- Future of kernel messaging in Jupyter Server (REST API vs. Websockets)
  - https://github.com/jupyter-server/jupyter_server/issues/901
  - https://github.com/jupyter-server/jupyter_server/issues/900

kevin-bates commented 2 years ago

July 14th, 2022

Name	affiliation	username
Steven Silvester	MongoDB	@blink1073
Zach Sailer	Apple	@Zsailer
Vidar T Fauske	JP Morgan Chase	@vidartf
Kiersten Stokes	IBM	@kiersten-stokes
Kevin Bates	IBM	@kevin-bates
Rahul Goyal	AWS	@rahul26goyal
Adam Patterson	UM	@adpatter

Agenda

Question for jupyter_server_ydoc:

Last week, it was mentioned that a contents_manager.save call happens 1 second after any change.
Expected that this would trigger the post_save_hooks, but this doesn't seem to happen.

Zach
- Discussion: File ID Service (jupyter_server#921, jupyterlab#12614)
Steve
- nbconvert failures in the prerelease build - should be fixed by nbconvert@7.0.0rc3 - Mistune APIs are changing again in the upcoming 3.0 release

Zsailer commented 2 years ago

July 21st, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Piyush
Kiersten Stokes	IBM	@kiersten-stokes
Vidar T Fauske	JP Morgan Chase	@vidartf
A. T. Darian	QuantStack	@afshin
Adam Patterson	UM	@adpatter

Agenda

Jupyter Events Update
- https://github.com/jupyter/jupyter_events/pull/2
- Public API changes
  - "categories" is no longer used for redacting sensitive information. Instead, the explicit key "redactionPolicies" is used to remove fields from an event.
    - To redact these fields, the EventLogger object uses a new trait, a list, called redactedPolicies. Any policies listed in this trait will be removed from emitted events
    - By default, the EventLogger emits all events unless they are explicitly redacted. This is a major difference than the previous version, where the EventLogger redacted all events unless they were explicitly listed.
    - You can prevent all events from being emitted by setting redactedPolicies="all"
  - allowed_schemas, allowed_categories, and allowed_properties, have been removed from the EventLogger.
    - All registered event types will be emitted.
    - To redact any data, use the "redactedPolicies" field
    - The expectation is that consumers will use filters in the logging handlers to listen for specific events.
  - redactionPolicies field is required on every schema and (nested) property object in a Jupyter event schema to pass validation.
  - EventLogger.record_event has been renamed to EventLogger.emit. (🚀)
Vidar:
- deque: We're applying nest_asyncio at the wrong time? Still some detective work to figure out what the correct fix is, but at least there is a workaround for now.
Adpatter:
- Interested in discussing (learning more about) authorization; if there is time.
  - https://github.com/jupyter-server/jupyter_server/pull/165
  - https://github.com/jupyter-server/jupyter_server/pull/825
- We discussed how to build a Jupyter server with custom auth using the IdentityProvider (to get a User) and Authorizator to get their permissions.

fcollonval commented 2 years ago

Last week, it was mentioned that a contents_manager.save call happens 1 second after any change.

At JupyterLab meeting, we got some discussion about the save UI for notebook. @williamstein mentioned that in CoCalc the interval to save the file on disk is customizable and set to 30s by default.

Zsailer commented 2 years ago

July 28th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Rick Wagner	UCSD	@rpwagner
Steve Silvester	MongoDB	@blink1073
Kiersten Stokes	IBM	@kiersten-stokes
Rahul Goyal	AWS	@rahul26goyal
Vidar Tonaas Fauske	JPMorgan Chase	@vidartf
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Kien Dang	NUS	@kiendang

Agenda

Require 2FA on jupyter-server org
- https://github.com/jupyter-server/team-compass/issues/32
- Plan:
  - Rick will trigger removal a backup account from a different org to see the email.
  - If the email is appropriate (spoilers, it is), then just enable 2FA for the org.
  - Users can have access restored if they want to contribute in the future and have enabled 2FA.
Zach
- Jupyter Events update
  - PR for Event Schema Registry
  - PR for redacting sensitive data
    - Should we only emit "notifications", not the full event blob?
      - only include event type and ID.
      - listener can then go back to the server for more authenticated detail about the event.
    - Consensus is that schemas shouldn't define what is sensitive. Admins should configure this.
    - Sensitive data is problematic when it leaves the application, but within the user's session in the application, it's okay to have it there. We should never record it.
    - We can't stop extensions from mishandling/recording this data, no matter how hard we try.
    - This shouldn't stop us from logging events to the user, since this could be useful to enrich their experience in extensions. We should inform/equipt extension authors with how to properly handle this data.
    - In a multi-user environment, we can provide filtering logic that uses Jupyter Server 2.0's auth APIs to ensure events are only emitted to authorized listeners.
  - Begin adding event schemas to Jupyter Server
    - This will help us determine what types of events might be considered sensitive.
Vidar
- https://github.com/jupyter-server/jupyter_server/pull/887/files
  - What should be the default behavior for enable/list?

kevin-bates commented 2 years ago

August 4th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Steve Silvester	MongoDB	@blink1073
Rahul Goyal	AWS	@rahul26goyal
Vidar Tonaas Fauske	JPMorgan Chase	@vidartf
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
David Qiu		@dlqqq
Piyush Jain	AWS	@3coins
Kiersten Stokes	IBM	@kiersten-stokes

Agenda

Steve
- Flake8 v5 woes
- Working on releaser upgrades
Darian
- A contributing hour proposal 🪄✨🎩
David Qiu
- yet another presentation of file ID service
- design doc: https://github.com/jupyter-server/jupyter_server/issues/940
- PR: https://github.com/jupyter-server/jupyter_server/pull/921
- Open questions from discussion:
  - Drop ID from model returned by GET Contents API. Build a separate API for requesting a file ID.
  - Migrate file ID service to a separate server extension given that ContentsManager no longer needs to rely on data returned from file ID service
  - Add some abstract base class for FileIdManager to guide developers on what public methods should be implemented
  - Windows getting dropped for Jupyter server 2.0? No need for Windows support?
Vidar
- Ready for review: https://github.com/jupyter-server/jupyter_server/pull/887

Zsailer commented 2 years ago

August 11th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Steve Silvester	MongoDB	@blink1073
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Kiersten Stokes	IBM	@kiersten-stokes

Agenda

Jupyter Events update
- v0.2.0 released last night
- Added a "Try it out" page to the docs powered by jupyterlite.
- Will integrate with Jupyter Server next
  - Replace jupyter_telemetry with jupyter_events: https://github.com/jupyter-server/jupyter_server/pull/862
  - Remove the singleton nature of the EventLogger.
  - Thread the event logger into the services.
- Question during discussion: could this work be backported to v1.0?
  - The file ID service is looking to switch to jupyter_events

dlqqq commented 2 years ago

@Zsailer Hey Zach, could you include a bit more of the discussion on the effort it would take to backport events to v1? Was on vacation and just got back.

kevin-bates commented 2 years ago

August 18th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Steve Silvester	MongoDB	@blink1073
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Kiersten Stokes	IBM	@kiersten-stokes
David Qiu		@dlqqq
Vidar T Fauske	JP Morgan Chase	@vidartf
Rahul Goyal	AWS	@rahul26goyal
Piyush Jain	AWS	@3coins
Jessica Xu

Agenda

Jupyter Enterprise Gateway 3.0.0b0 is available.
PR: Can't open a directory named checkpoints (or trust)
- https://github.com/jupyterlab/jupyterlab/issues/12408
- https://github.com/jupyter/notebook/issues/5569
Maybe bite the bullet and just make sure we apply nest_asyncio patch at known good point: https://github.com/jupyterlab/jupyterlab/issues/11934
- When patchin loop while asyncio's _run_once is processing a long queue, the reentrancy means two places are popping from the same queue...
David Q: File ID Service maybe should not use the event bus
Jupyter Events update

vidartf commented 1 year ago

August 18th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Steve Silvester	MongoDB	@blink1073
A. T. Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Kiersten Stokes	IBM	@kiersten-stokes
David Qiu		@dlqqq
Vidar T Fauske	JP Morgan Chase	@vidartf
Rahul Goyal	AWS	@rahul26goyal
Piyush Jain	AWS	@3coins
Jessica Xu

Agenda

Jupyter Enterprise Gateway 3.0.0b0 is available.
PR: Can't open a directory named checkpoints (or trust)
- https://github.com/jupyterlab/jupyterlab/issues/12408
- https://github.com/jupyter/notebook/issues/5569
Maybe bite the bullet and just make sure we apply nest_asyncio patch at known good point: https://github.com/jupyterlab/jupyterlab/issues/11934
- When patchin loop while asyncio's _run_once is processing a long queue, the reentrancy means two places are popping from the same queue...
David Q: File ID Service maybe should not use the event bus
Jupyter Events update

Zsailer commented 1 year ago

August 25th, 2022

Name	affiliation	username
Vidar T Fauske	JP Morgan Chase	@vidartf
Steve Silvester	MongoDB	@blink1073
Piyush Jain	AWS	@3coins
David Qiu	@dlqqq
Rahul Goyal	AWS	@rahul26goyal
AT Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Zach Sailer	Apple	@Zsailer
Kiersten Stokes	IBM	@kiersten-stokes
Kevin Bates	IBM	@kevin-bates

Agenda

Steve
- nbconvert 7.0 release
  - mistune > 1 is still causing problems
    - Had to pin our docs build
Vidar
- Change to jupyter_client: jupyter_client#822
- Makes it so that the default config doesn't use run_sync (AFAICT), but doesn't deal with what happens when a sync manager etc is used which uses run_sync.
- Either do something like jupyter_client#823 to try to ensure earlier patching, AND/OR patch nest_asyncio in jupyter-server when initing the ioloop (put it behind a flag?)
- Request for review: https://github.com/jupyter-server/jupyter_server/pull/887
Piyush
- File Id Manager: https://github.com/jupyter-server/jupyter_server/pull/921
  - Help with setting up a new repo under jupyter-server to setup as a separate extension (https://github.com/jupyter-server/team-compass/issues/33)
  - Does these docs apply to us? https://jupyter.org/governance/newsubprojects.html
David Qiu
- server extension dependency management? :thinking_face:
  - https://github.com/jupyter-server/jupyter_server/issues/178#issuecomment-582072588
  - the order in which extensions are loaded sometimes matters
  - we could use the "linking" phase of the server extension logic to resolve extension loading order and dependency.
Zach
- Jupyter Events update
  - Modifiers and listeners merged this week
  - Removed version from the public API
  - Moving jupyter_events into Jupyter Server

kevin-bates commented 1 year ago

September 1st, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Piyush Jain	AWS	@3coins
Andrii Ieroshenko	AWS	@andrii-i
Vidar T Fauske	JPMorgan Chase	@vidartf
Kiersten Stokes	IBM	@kiersten-stokes
AT Darian	QuantStack	@afshin
Rahul G	AWS	@rahul26goyal
Luciano Resende	Apple	@lresende

Agenda

Zach
- Jupyter Events update
  - Jupyter Server now uses jupyter_events
  - Default ContentsManager now emits events.
- File ID server extension vote
  - 7/14 yes, 7/14 no response
  - Let's open the repo
- Jupyter Server 2.0 release
  - Jupyter Server 2.0.0b1 has been released
  - No leftover tasks listed in milestone or roadmap.
  - Are we missing anything else?
    - Double check we finished everything around identity and auth.
  - Let's aim to cut an RC next week.
  - Release checklist:
    - Jupyter blog+discourse post
Piyush
- Headless/Scheduled Notebooks extension
  - https://github.com/jupyter-server/jupyter_server/pull/957
  - How do we proceed to create a new repo for this? Should I open a team compass issue?
    - New team compass issue opened: https://github.com/jupyter-server/team-compass/issues/34
Darian
- @lumino/polling async iterator and events system
- Question about voting in GitHub discussions in a private repo for @jupyter-server/council
  - Repo created.
- Question about list of Jupyter Subprojects viz. Jupyter Telemetry

Zsailer commented 1 year ago

September 8th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
David Qiu	@dlqqq
Kevin Bates	IBM	@kevin-bates
Frederic Collonval	QuantStack	@fcollonval
Kiersten Stokes	IBM	@kiersten-stokes
Piyush Jain	AWS	@3coins
Andrii Ieroshenko	AWS	@andrii-i
AT Darian	QuantStack	@afshin
Steven Silvester	MongoDB	@blink1073
Vidar T Fauske	JP Morgan Chase	@vidartf

Agenda

Pagination support in the contents service
- Practical issue for filebrowser in JupyterLab: https://github.com/jupyter-server/jupyter_server/issues/538
  - In that issue, the calls of os.lstat can be a bottleneck due to the large number of files.
- What is the problem definition? (memory, rendering time, both, other?)
  - Primarily memory - only want "the view" in memory.
  - Secondary is rendering time due to the large increase of DOM nodes to render the full list.
  - Implemented pagination + filtering raises questions of how do you know how many pages there are for a filter query?
  - Can query results be kept in a server-side cache?
  - David Q suggestion: only run query (e.g. call os.stat, sort, filter, etc.) on the first page, cache it in a persistent store (e.g. SQLite), and then every subsequent page is a LIMIT OFFSET query on the cached results
    - Would need to distinguish a fresh GET vs. a traversal GET when the latter targets page=1 (noted here).
- Live trial of Vidar:
  - 3k+ entries in contents manager, with gzip: 800 Kb -> 91 kB
  - We should probably consider if JupyterLab can use browser caching (etags should be reliable?). Current it uses no-cache header. i.e. so that we don't resend it on polls if there are no server-side changes (most calls).
Zach
- Jupyter Server Council Updates
  - Vote status on "New repo for Headless/scheduled Notebook Execution"
  - New Team member!
- Jupyter Server RC
  - Bump Jupyter Server version in JupyterLab Server and JupyterLab
David Q
- tracking issue for backporting events to server v1: https://github.com/jupyter-server/jupyter_server/issues/960
  - Address https://github.com/jupyter/jupyter_events/issues/26
  - Release Jupyter Events
- publishing to read the docs and pypi registry?
Piyush
- Why is JL4 not using JServer 2.0?
  - [name=frederic] Feel free to open the PR - but yes it will :wink: :thumbsup:
  - We should look to add an item for releasing 3.5 with JServer 2 The login handler change is probably the only one that can bring some issue for user overriding it We need to check that no deprecated APIs are used by JLab 3.x - just to be sure.
Contributing Hour Topics
- Publishing server extension to readthedocs
- Publishing server extension to PyPI.

Zsailer commented 1 year ago

September 15th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Vidar T Fauske	JP Morgan Chase	@vidartf
David L. Qiu		@dlqqq
AT Darian	QuantStack	@afshin
Piyush Jain	AWS	@3coins
Sylvain Corlay	QuantStack	@SylvainCorlay
Steven Silvester	MongoDB	@blink1073
Kiersten Stokes	IBM	@kiersten-stokes
Rahul Goyal	AWS	@rahul26goyal
Matt Henderson	LBNL	@mlhenderson
Andrii Ieroshenko	AWS	@andrii-i

Agenda

SSC Representatives
- Nominate by next week (Google group)
- Vote the following week
- New governance goes into effect by October 2022
Jupyter Server 2.0
- RC released this past week
- Need to write a Jupyter Blog post
- Testing with JupyterLab
  - JupyterLab 3.5 will allow Jupyter Server 2.0
  - JupyterLab 4.0 will require Jupyter Server 2.0
  - Hold off on final release until JupyterLab makes releases allowing folks to test JS 2.0 with JLab.
Jupyter Client 8.0 plan
- Jupyter Events merged: https://github.com/jupyter/jupyter_client/pull/832
- Working on adding jupyter-events, removing nest-asyncio, and adding a monitor that pulls in some logic that is currently in jupyter_server.
Possible contribution to resolve the message buffer replay across re-opened tabs/sessions.
- Currently implemented in extensions (front and back) and a custom ContentsManager that would be folded into respective cores
- Concerned about spending significant time to introduce something that RTC should be addressing
- Any update on RTC's release? (Slated for Lab 4.0 [4Q22], may be back-ported to 3.5)
PR to fix preferred_dir for non-FS contents manager:
- https://github.com/jupyter-server/jupyter_server/pull/983
- The logic for being backwards compatible ended up a bit convoluted, so if anyone has suggestions for simplyfing, please help :)
- NOTE: os.path.join("C:\\foo", "G:\\bar") --> "G:\\bar"...
David Q: announcement: new on-call engineer at AWS starting next week
- responsibilities: help with devtooling and ops, review PRs
- proposal for initial work: remove redundant linting / CI steps (flake8/tbump/precommit step)
  - Switch to "hatchling version"
- existing opportunities for improvement in these areas?
  - Maintainer Checklist
  - Add a label for issues/PRs for "maintainer help needed"
Sylvain C: kernels SSC No pre-existing meeting or team compass before the new governance, but a lot of ongoing activity, and a lot of work ahead that will require cross-project coordination
- lite kernels standardization
- JEPs
- xpub https://github.com/jupyter/enhancement-proposals/pull/65
- kernel handshake https://github.com/jupyter/enhancement-proposals/pull/66
- parameterized kernelspecs https://github.com/jupyter/enhancement-proposals/pull/87
- kernel info in the control channel https://github.com/jupyter/enhancement-proposals/pull/82
- async kernels
- kernel status in lab and other frontends.
Matt H: question/feedback requested about PR approach related to resource_dir template string
- resource_dir is not returned by the server, so for certain contexts you can't find the path to the kernel
- #818 adds a resource_dir field to server response
- #932 replaces the template string prior to server response with kernelspec
- We'll commit to reviewing these PRs this week and get this into 2.0.

jasongrout commented 1 year ago

New governance goes into effect by October 2022

I'm sure there was more context in the actual meeting, but just to clarify for those reading the notes: the overall new Jupyter governance (Software Steering Council, Executive Council) transition is planned for before the end of the year (probably in December). For October, we are planning for subproject councils to be in place and functioning.

Zsailer commented 1 year ago

September 22nd, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Steven Silvester	MongoDB	@blink1073
Kevin Bates	IBM	@kevin-bates
Kiersten Stokes	IBM	@kiersten-stokes
AT Darian	QuantStack	@afshin
David Brochart	QuantStack	@davidbrochart
Piyush Jain	AWS	@3coins
Chia-Wei Chen	Pinterest	@chiawchen
Rahul Goyal	AWS	@rahul26goyal	Sylvain Corlay	QuantStack	@ SylvainCorlay

Agenda

Steve
- Working on removing nest-asyncio from the stack, for compliance with Python 3.12 changes in asyncio.
  - Client PR
  - Server PR
Kevin
- Nearing EG 3.0 GA
- Updating remote provisioners with "recent" EG fixes, would like to move into jupyter-server once CI/build is updated
- Thinking lots about kernel parameterization, discovery in particular
  - Related JEP: Parameterized kernel specs proposal
Zach
- Kernel state bug
  - Busy kernels show an inaccurate state in JupyterLab on a refresh
  - JupyterLab is watching IOPub status messages
  - Nudging logic in Jupyter Server triggers IOPub messages from both control AND shell channel.
    - Shell channel is "busy"
    - Control channel is "idle"
    - The last "status" that comes through is "idle" (because of control), but the kernel shell channel is the actual state of the kernel.
  - What does "execution_state" mean for the async kernel?
    - Each execution is a separate task. If each task is publishing IOPub statuses, determining the "full" status is tricky.
- Jupyter Server 2.0rc
  - Waiting for a JupyterLab 3.5 branch to be created to backport https://github.com/jupyterlab/jupyterlab/pull/13068
  - Once we can release JupyterLab 3.5 with Jupyter Server 2.0, we can move into a final release.
  - One more RC release?
    - https://github.com/jupyter-server/jupyter_server/pull/985
    - https://github.com/jupyter-server/jupyter_server/pull/981
Sylvain
- Questions about JupyterHub patching in Jupyter-server
  - Does the Authorizer/Identity Provider enable us to drop monkeypatching in JHub?
    - YES! :rocket:
  - minrk https://github.com/jupyterhub/jupyterhub/pull/3888/files

kevin-bates commented 1 year ago

September 29th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Steven Silvester	MongoDB	@blink1073
Kiersten Stokes	IBM	@kiersten-stokes
Andrii Ieroshenko	AWS	@andrii-i
Rahul Goyal	AWS	@rahul26goyal
David Brochart	QuantStack	@davidbrochart

Agenda

Zach
- SSC Nominations. Finish nominations this week.
- Question about extensions that span multiple layers. Where do they go?
  - We've tried "contrib" repos—more orgs aren't great...
  - Hypothetical "Jupyter-plugins" org.
    - Not a Jupyter subproject; i.e. no SSC rep
    - Not officially governed by Jupyter
    - Has representation from people with Jupyter subprojects team membership.
    - Inherits some processes from Jupyter governance, i.e. team-compass, team-membership, voting to accept outside repos.
    - Set expectations for plugin repos.
      - Only accept repos that show long term support.
      - Must inherit Jupyter's BSD-3 license.
  - Why use a separate non-Jupyter governed repo?
    - One layer of separation from Jupyter core components; sets the users expectations.
    - Still reviewed and evaluated by representation from Jupyter subproject members.
      - Users of the plugins can "trust" it more than plugins under a single user's namespace
Steve
- Removing pending-kernels from client 8?
  - We can provide the same behavior by wrapping the Kernel Manager and tracing method calls, and using our own MultiKernelManager in Server v2. Server v1 would pin to client<8.
- Releaser changes - experimental support for Releaser v2 that is run from Jupyter Server itself using shared credentials - requires admin write access to the repository. The intent is to repeat this pattern across most of the Jupyter repos.

ellisonbg commented 1 year ago

Can you give more details about what is meant above by "extensions that span multiple layers"?

On Thu, Sep 29, 2022 at 10:03 AM Kevin Bates @.***> wrote:

September 29th, 2022 Name affiliation username Zach Sailer Apple @Zsailer https://github.com/Zsailer Kevin Bates IBM @kevin-bates https://github.com/kevin-bates Steven Silvester MongoDB @blink1073 https://github.com/blink1073 Kiersten Stokes IBM @kiersten-stokes https://github.com/kiersten-stokes Andrii Ieroshenko AWS @andrii-i https://github.com/andrii-i Rahul Goyal AWS @rahul26goyal https://github.com/rahul26goyal David Brochart QuantStack @davidbrochart https://github.com/davidbrochart Agenda

Zach

SSC Nominations. Finish nominations this week.

Question about extensions that span multiple layers. Where do they go?

We've tried "contrib" repos—more orgs aren't great...

Hypothetical "Jupyter-plugins" org.

Not a Jupyter subproject; i.e. no SSC rep

Not officially governed by Jupyter

Has representation from people with Jupyter subprojects team membership.

Inherits some processes from Jupyter governance, i.e. team-compass, team-membership, voting to accept outside repos.

Set expectations for plugin repos.

Only accept repos that show long term support.

Must inherit Jupyter's BSD-3 license.

Why use a separate non-Jupyter governed repo?

One layer of separation from Jupyter core components; sets the users expectations.

Still reviewed and evaluated by representation from Jupyter subproject members.

Users of the plugins can "trust" it more than plugins under a single user's namespace

Steve

Removing pending-kernels from client 8?

We can provide the same behavior by wrapping the Kernel Manager and tracing method calls, and using our own MultiKernelManager in Server v2. Server v1 would pin to client<8.

Releaser changes - experimental support for Releaser v2 that is run from Jupyter Server itself using shared credentials - requires admin write access to the repository. The intent is to repeat this pattern across most of the Jupyter repos.

— Reply to this email directly, view it on GitHub https://github.com/jupyter-server/team-compass/issues/15#issuecomment-1262557417, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAGXUHJNKNXPDDA7C63LOLWAXDVDANCNFSM5LMUIYJQ . You are receiving this because you were mentioned.Message ID: @.***>

-- Brian E. Granger

Senior Principal Technologist, AWS AI/ML @.***) On Leave - Professor of Physics and Data Science, Cal Poly @ellisonbg on GitHub

Zsailer commented 1 year ago

Hey @ellisonbg, in the discussion today, we were thinking about extensions that have both server-side and client-side components/extension that are closely coupled together. This could be expanded to plugins that work in a JupyterHub context—where it requires single-user server extension, client UI extension, and a JupyterHub-managed "service" to coordinate the UX.

I can open a separate thread for future discussion.

Zsailer commented 1 year ago

@ellisonbg I opened this issue for further discussion: https://github.com/jupyter-server/team-compass/issues/38

kevin-bates commented 1 year ago

October 6th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Piyush Jain	AWS	@3coins
Frederic Collonval	QuantStack	@fcollonval
Kiersten Stokes	IBM	@kiersten-stokes
Andrii Ieroshenko	AWS	@andrii-i
Jessica Xu	Cal Poly	@jess-x

Agenda

Frederic:
- Annoucement JupyterLab 3.5.0 beta 0 has been released. So it is now possible to test JupyterLab against Jupyter Server 2:
```
pip install --pre jupyterlab==3.5.0b0
```
I did some quick tests, it works (files are listed, can be opened, kernels execution is fine, terminals opens).

ETA to release the final version of jupyterlab 3.5.0 and jupyterlab-server is in 2 weeks. But this does impact the server release.
Zach
- SSC Vote
- Where should Jupyter plugin projects live?
Steve
- Client nest-asyncio removal - will release nbclient and nbconvert after this call, and the client PR will be ready to merge
- Releaser Updates - Currently in use on Server, need to update security handling - see releaser PR. I will make a PyPI user and associated email for our GitHub org, and we will use scoped PyPI tokens for publish. We'll also need an npm user.
Kernel state bug (update)
- Background
- Kernel Manager refactoring
  - One set of ZMQ sockets per KernelManager; multiple kernel clients with their own Session and ZMQ Channels.
  - Remove dependency on tornado's ioloop APIs (no more IOLoopKernelManager); replace the ZMQStream object with an asyncio Task that's listening for messages on the ZMQSocket.

Zsailer commented 1 year ago

October 13th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Kevin Bates	IBM	@kevin-bates
Andrii Ieroshenko	AWS	@andrii-i
Sylvain Corlay	QuantStack	@SylvainCorlay
David Brochart	QuantStack	@davidbrochart
Thorsten Beier	QuantStack	@derthorsten
AT Darian	QuantStack	@afshin
Johan Mabille	QuantStack	@JohanMabille
Kiersten Stokes	IBM	@kiersten-stokes
Jessica Xu	Cal Poly	@jess-x
Antoine Prouvost	QuantStack	@AntoinePrv
Frederic Collonval	QuantStack	@fcollonval
Michal Krassowski	(Quansight)	@krassowski

Agenda

Frederic (need to drop after 15min):
- JupyterLab 3.5.0rc0 has been released (lift Jupyter Server version constrain) - ETA for final is Oct. 20th.
- JupyterLab server 2.16.0 has been released (lift Jupyter Server version constrain)

Jupyter kernels

Antoine : xeus-octave
- Binder link
- Octave Jupyter Kernel build in C++
- Highlights
  - Plotly integration
  - Latex-based rich output of matrices
  - Widget integration
Thorsten & Sylvain : long-running kernels
- Connecting to kernels that are not started/stopped by the main Jupyter instance
- The kernels persist beyond the lifetime of the Jupyter application.
- We would connect to them by just knowing their connection info.
- Ideally they would show up in the "running kernels" section of JupyterLab
- Ideas discussed:
  - Use some of the existing APIs in Jupyter client to connect to these kernels. Look into kernel clients
  - Use custom provisioner to discover a kernel, but maybe no-op some of the lifecycle calls (e.g. start/stop)?
  - There are some rehydration work happening in enterprise gateway that might be useful for connecting to a kernel that already exists.

Jupyter FileID:

GET /api/fileid handler.
- Discussed concern about creating an index on a fetch call.

Jupyter SSC Voting

Voting happening this week. Check your inboxes

Jupyter Releaser

Created a Jupyter Server Bot with write credentials
Shared PyPI repo-scoped access token for publishing jupyter server
Releaser will pull the Github API to check if the user doing the release has admin permission for publishing.

Clean up old issues on team-compass

Did you create / answer an issue on team-compass? Can it be closed? It will be easier to check for votes etc if things are kept up to date :+1:

kevin-bates commented 1 year ago

October 20th, 2022

Name	affiliation	username
Zach Sailer	Apple	@Zsailer
Steve Silvester	MongoDB	@blink1073
Kevin Bates	IBM	@kevin-bates
Kiersten Stokes	IBM	@kiersten-stokes
Piyush Jain	AWS	@3coins
AT Darian	QuantStack	@afshin
Andrii Ieroshenko	AWS	@andrii-i
Rahul Goyal	AWS	@rahul26goyal
Jessica Xu	Cal Poly	@jess-x
Theodore Aptekarev	OpenBB	@piiq

Agenda

Zach
- 2.0 release plans
- SSC stuff
- (Non-breaking changes to) Kernel Websocket Architecture
  - Add a new configurable "kernel websocket client" class to ServerApp that owns all the logic underneath the ZMQChannelsHandler (websocket handler).
    - The websocket handler would only include the public Tornado methods. Everything else belongs under the configurable class.
    - In my current design (not public yet), the client class includes the following public methods:
    - ```
    class KernelWebsocketClient(...):
    
        session: jupyter_client.session.Session
    
        def connect():
            ... 
    
        def disconnect():
            ... 
    
        def handle_incoming_message(ws_msg):
            ... 
    
        def handle_outgoing_message(zmq_msg):
            ...
    
        def send_status_message(state):
```
- Why?
  - Enables folks to customize the logic under the websocket handler.
  - Today, you can't remove the websocket handler in favor of your own.
  - You can write a new websocket handler and inject it before the core websocket handler, but that's a hack.
- Name is hard. What should we call this (sorry to bikeshed here)?
  - Jessica suggested: Kernel Message Broker
- Finish before 2.0 release.
- Steve
  - Releaser Updates
    - We are now using releaser on server repo itself. We have bot accounts for GitHub, PyPI, and npm. We're using a scoped PyPI token for server. The npm account has access to the @jupyter-server npm org, which right now only affects resource-usage (which is not yet using releaser).
  - Asyncio Cleanup
    - Incorporate changes Min had made earlier
    - Required a fork of pytest-tornasync to handle new asyncio
    - Most of the fixtures are now async
    - Since it is a breaking change to the fixtures (and a couple utility functions), it should be in 2.0
    - Deprecate MappingKernelManager and all sync managers (non-asyncio) in 2.0?
    - Zach will work on async KernelSpecManager
- Darian
  - Making file ID service opt-in for contents managers: https://github.com/jupyter-server/jupyter_server_fileid/issues/19

jupyter-server / team-compass

Meeting Notes 2022 #15

Hello everyone,

Meeting Notes

January 6th, 2022

Attendees

Agenda + Minutes

January 13th, 2022

Attendees

Agenda + Minutes

January 20th, 2022

Attendees

Agenda + Minutes

January 27th, 2022

Agenda + Minutes

February 3rd, 2022

Agenda + Minutes

February 10th, 2022

Agenda + Minutes

February 17th, 2022

Agenda + Minutes

February 24th, 2022

Agenda + Minutes

March 3, 2022

Agenda + Minutes

March 10, 2022

Agenda + Minutes

March 17th, 2022

Agenda + Minutes

March 24th, 2022

Agenda + Minutes

March 31st, 2022

Agenda + Minutes

April 7th, 2022

Agenda + Minutes

April 14th, 2022

Agenda + Minutes

April 21st, 2022

Agenda + Minutes

April 28th, 2022

Agenda + Minutes

May 5th, 2022

Agenda + Minutes

May 12, 2022

Agenda + Minutes

May 19, 2022

Agenda + Minutes

May 26, 2022

June 2nd, 2022

June 9nd, 2022

June 16th, 2022

June 23rd, 2022

June 30th, 2022

July 7th, 2022

Agenda

July 14th, 2022

Agenda

July 21st, 2022

Agenda

July 28th, 2022

Agenda

August 4th, 2022

Agenda

August 11th, 2022

Agenda

August 18th, 2022

Agenda

August 18th, 2022

Agenda

August 25th, 2022

Agenda

September 1st, 2022

Agenda

September 8th, 2022

Agenda

September 15th, 2022

Agenda

September 22nd, 2022

Agenda

September 29th, 2022