Add some custom ipykernel magics for the Jupiter notebook environment

[Hamada-Distributed]

Jupiter notebooks are a common python dev environment. It would be really useful to these devs if we provided a few ipykernel magics.

Something that could enable a js cell or something else like this would be nice.

[bollwyvl]

Hi there, Jupyter dev here! If you have any questions about this, happy to help!

enable a js cell

At present, the languages are fairly strictly isolated by kernel: you get a code cell, and all the cells will get that same content. So the magic is definitely the right play. The js magic already exists, so might want to be ginger there (folk expect to actually manipulate the DOM).

What is potentially possible is making a new grammar, implemented by a wrapper of ipykernel, that relaxed the distinction somewhat. We us that in e.g. https://github.com/deathbeds/pidgy/ or https://github.com/robots-from-jupyter/robotkernel or even more exotic things like https://github.com/minrk/allthekernels ... but if this just ends up inventing a new, non-portable syntax (instead of the magics' % and %) then it might not improve the experience much.

Some broader Jupyter :heart: pythonmonkey stuff:

With regards to the custom kernel stuff, this could drastically simplify running some language servers, as nodejs gives non-webdev Jupyter users (and their sysadmins) no end of trouble. We're of trying to get more things running in the browser but that's another story...

JupyterLite could benefit highly from this project, as we'd be able to do some conformance testing outside of a heavyweight browser. A longer goal might be some kind of common syntax/API with pyodide's js module: seems unlikely one could run them both together simply without an incredible amount of turtle shaving, so agreeing on a js API that worked in the browser and server would be rather lovely.

And finally: a big enabler of Jupyter's cross-language experience is the conda-forge ecosystem... I've started looking into packaging pythonmonkey, but it is of course a bit of a slog through the build chain. A key thing there, from my cursory look, is the level of vendoring (usually without attribution). To really get this into conda-forge, it would likely need some patches to use the as-installed spidermonkey libraries by normal means instead of vendoring, while pminit would need to collect all of the license files shipped in node_modules. While this is doable downstream, having some switches in this repo would substantially improve the situation.

[wesgarland]

@bollwyvl Thanks very much for this. It might be useful to have a Zoom call or something some day about this.

Pythonmonkey offers a very predictable JS environment from Python. It is nowhere near as complete as Node.js, but in many ways, that's actually the point. :) I started this project because we had Python devs exchanging data with Node.js over unix-domain pipes in order to access Node libraries, and it was really ugly, time-consuming, and fraught with peril. Our local folks use both Jupyter notebooks and Google Collabs; not heavily, but we are working on data-science related tech, so they come up regularly.

PythonMonkey gives a path to accessing JS libraries in a way which is not as memory-intensive (backing stores are shared across interpreters) and "live". So, when you change a Dict in Python, the corresponding object "changes" in JS because it is the same object. (Or, rather, the either Dict's backing store is a JS Object or the JS Object's backing store is a Dict, depending on how it was declared)

Our current project is to make PythonMonkey run a library of ours called dcp-client. This library can run in a browser environment and does network I/O with socketio over XMLHttpRequest. I expect that will be shipping in less than a month, and that will mark our "1.0" release. We have recently resolved a number of stability issues and other bugs that were getting in the way, but things started to come together last week.

I'm not a Python guy, my expertise is UNIX systems dev + low-level JS. So some of what you've mentioned here is over my head. That said, I'm also very interested in helping you package PythonMonkey and we can put some cycles into making changes that will make this easy for you. If you can, for example, guide the creation of some detailed PRs I can put developer resources into them.

Sorry I missed this in August, I have a thousand things going on and frankly, I'm not very good with email and whatnot. If you'd like to set up a real-time chat (I am very eager to do so), feel free to text the square root of 260351034798610890304, you'll have my immediate attention :). I'm in Canada/Eastern, GMT-5. My email address is wes@distributive.network.