Create a data science "working group"

[eamodio]

Refs: #93604

There was a request from the March Extension Authors call to create a sort of working group of data science extension authors (Julia, R, Python, etc) to share and collaborate on cross-cutting requirements and for VS Code to learn in be informed from the baseline infrastructure required for VS Code to support innovation in the data science space.

/cc @davidanthoff

[Ikuyadeu]

FYI @andycraig and @renkun-ken are main great developers in vscode-r

[RandomFractals]

I created a dedicated data-science channel in VS Code Dev Community Slack for dataSci/dataViz extension authors to collaborate.

[andycraig]

It seems like VSCode Notebooks might be a good topic for a data science working group https://github.com/microsoft/vscode/issues/91987 https://github.com/microsoft/vscode/issues/93265

I’m basically waiting for VSCode Notebooks before I do any major work on R Markdown support over at vscode-r and I think @davidanthoff is doing the same for Julia.

I’m hoping VSCode Notebooks will support (or allow us to support) the R Markdown style R users are familiar with as well as the Jupyter Notebook style.

EDIT: Fixed issue links.

[jlperla]

If this is moving forward, may I also point out https://github.com/executablebooks/jupyter-book @choldgraf @mmcky

[choldgraf]

Just a note that the flavor of markdown that we're supporting with Jupyter Book is heavily inspired by RMarkdown. It aims to be a bit more language-generic than RMarkdown, but has many of the same syntax patterns (e.g., ```{}).

I think that it is still too young now to be considered a standard, but I hope that we can grow the adoption throughout at least the Sphinx user community (https://myst-parser.readthedocs.io/).

In order to create the myst-parser, we also created a Python port of the very popular (and pluggable) javascript markdown parsing library: markdown-it. It is called markdown-it-py and one of our goals was having a Python library that could be mirrored with the Javascript implementation so that it was easy to mirror extensions in both Python and Javascript. I think that's the first step towards supporting MyST markdown in both Jupyter interfaces like notebook and lab, as well as notebook interfaces like vscode.

Finally, as one of team members is a big fan of vscode, we've already got a myst markdown extension for vscode that uses an LSP implementation of myst! :-)

[RandomFractals]

since we are sharing, on the JavaScript side, you might find this Observable JavaScript markdown extension relevant: https://marketplace.visualstudio.com/items?itemName=GordonSmith.observable-js

It uses custom file extensions .ojs and .omd for creating observable js notebooks.

Also, I am working on vscode extension to bring Observable JS to vscode with options to load and render those notebooks for now. Hoping to get liking, search and code cells nav done this month:

https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.js-notebook-inspector

P.S.: do join that Slack channel I created in the unofficial vscode devs workspace for sharing more where native notebooks api is headed.

I did check out the proposed version. I think it's close enough for supporting both Jupyter and Observable JS style notebooks, but it's not shipping with vscode for the general public yet, so I think our best bet is just to iterate on custom extensions like those listed above and lang. servers.

[davidanthoff]

Sorry that I dropped of the radar after suggesting this in the first place :) Some very unexpected other stuff kept me away from this.

I think from the Julia side of things, we are super excited about the notebook stuff, and pretty much everything around that is very clear to us and we really like the design etc.

I think what I had mostly in mind when I brought this up is whether there are other areas for the various language extensions to share code. There seem to be whole bunch of other UI elements that pretty much all the data sciency language extensions are adding: 1) workspace explorer (i.e. list of variables in the REPL and their values), 2) some sort of grid viewer for tabular variables in the REPL, 3) something to show plots and probably other stuff.

So I guess one question is whether the best approach for this is that a) each extension rolls their own, or b) VS Code core adds support for some of these UI elements over time (in the same way in which they are adding support for notebook UIs right now) that extensions can utilize, or c) whether there could be a model that some of the language extension say share some common code for say a plot gallery in a third extension.

I think for c), I could imagine a world where there is one high quality "data science core" extension that say the Julia, R and Python extension all depend on. That "data science core" extension might provide a UI for the elements that I mentioned, and then the various language extension would hook via some API that such an extension provides, into it.

But not sure whether that (or something else) would be a good model. Mainly it just seemed to me that many of the data sciency language extensions right now seem to re-implement very similar UI concepts, and to figure out whether there should be some coordination or not.

[RandomFractals]

I agree with @davidanthoff There is definitely more opportunity for generic UI api setup and reuse when it comes to custom web view editors, notebooks, graph/viz galleries and grid data viewers.

Since I've created a few of them over the past year that I see many devs using, especially data preview ext., I think it would be worthwhile revisiting this UX topic we touched on in our initial 3rd party ext. devs discussion: https://github.com/microsoft/vscode/issues/94741#issuecomment-612398069