Integration with Jupyter ?

[aaelony]

Gorilla-repl is awesome, would more traction (user base) happen if it was coordinated with the Jupyter project? It would be great to be able to be in Jupyter and select Clojure from among the other choices offered (python, R, spark, erlang, F#, etc...) i.e. Jupyter "Evolved from the IPython Project. The language-agnostic parts of IPython are getting a new home in Project Jupyter"

https://jupyter.org/

Is this an interesting or desirable topic?

regards, Avram

[aaelony]

https://try.jupyter.org/

[JonyEpsilon]

Hi Avram,

yes I think there is an interesting discussion to be had here. I can see reasons both for and against for the idea of integrating Gorilla into Jupyter.

On the "against" side, the main reason is that I see Gorilla as a distinct thing than Jupyter. There are a few design decisions that set them apart, but the main one is how the rendering works. Gorilla's renderer is very strongly tied to the idea of immutable data, whereas Jupyter has a model that more naturally fits python (and, is flexible enough to work with a wide range of languages). I think the Gorilla renderer is more elegant (of course I do!) and more powerful in a sense: in that you can compose rendered items as naturally as you compose data structures; and that it is extremely easy to extend in a way that is very natural to Clojure. The Jupyter rendering model is more generally applicable to languages that aren't data based, and less restricted, so they can do cool things like interactive widgets ... although it does come at some cost in terms of simplicity.

The other thing I think is nice about Gorilla that sets it apart from Jupyter is that it is dedicated to doing one thing - providing a notebook for Clojure - and as a result can be very simple and lightweight; can be closely integrated with other Clojure tooling; and has zero install process for an already-set-up Clojure user. This is not a criticism of Jupyter, to be clear, just to note that the different design goals end up with different designs. And from another perspective, this is an anti-feature ... you might say the great thing about Jupyter is that isn't dedicated to just doing one thing!

On the "for side", there are many reasons. Jupyter is more and more establishing itself as the de-facto open notebook. They're doing an excellent job of pushing towards being a cross-language notebook solution. The user experience is getting much more polished with every version. And their adoption is really strong, quite deservedly. I think it would be great to have first class Clojure support in Jupyter - it would definitely have broader appeal than Gorilla, and there's much value in having people develop expertise with a common tool. I haven't used the IClojure plugin yet, so I don't know what state that is in. But as far as I know it doesn't have much support for plotting, graphics etc yet.

So, an obvious question would be whether it's possible to take Gorilla REPL's renderer, and plug it in to Jupyter in a useful way? And whether it would be possible to have the same close integration of Jupyter with the rest of the clojure ecosystem (leiningen, cider etc) ... perhaps through a lein-jupyter plugin.

I should note here that one of the IPython devs reached out to me to start a discussion, but I got caught up with other projects and dropped the ball (for shame, sorry Matthias!). I'll mention him in - @Carreau - in case he wants to contribute, or you want to reach out to him.

So, to summarise my long ramble. Yes I think there's good reason to think about how Clojure is used in Jupyter, and it would make sense to try and integrate Gorilla's strengths into any such effort. I'd certainly be interested to think about whether Gorilla style rendering could be integrated into IClojure, or something similar.

Jony

[Carreau]

Hey, thanks for pinging me, no need to apologize for being caught up by other projects, we all are.

I think that all point are valid, I haven't particularly looked into the rendering engine used here, but I believe you in the term of composability. Side note, I'm not a huge fan of widgets either, but now they should be "just" a plugin on Jupyter. We do our best to make thing as agnostic and flexible, but Jupyter was designed first with python in mind we probably do not have the best framework.

I'm a bit disappointed not not have more functional programmers in the core team, and would also like to have more people from the javascript world to work on the UI. I think that IHaskell made a awesome job as pushing what you can do to the limit and I would love to see Andrew (IHaskell author) getting funding to do that full time.

Of course we (The core team) could investigate all kernels and separate project ourself. It's not a lack of willingness to get feedback or investigate what other kernel and project are doing, but my time at least start to dangerously approach days of only responding to email, and not actually getting things done. Gorilla-REPL is I think one of the place where we could learn from immutability.

Hopefully, we start to have some funding to actually get some work done on breaking the various component in smaller pieces, so it might be easier to just reuse pieces of our frontend, or backend, or network protocol, so it should just make writing separate project that take a different approach easier, and free up time for people like @JonyEpsilon to actually defend their idea, and get a product as polished as the Jupyter notebook much more easily (Not to say that Gorilla is not polished, but we probably have more manpower to nick pick on detail even more). So we are open to project like gorilla REPL, we are open to discussion, share ideas, help integrating or even get constructive discussion on why we disagree on fundamental point. We definitively want life of people that make this kind of things easier, not harder.

One of the things we are relatively strict about though, is that as most of the core team of Jupyter is from a Science/Academic background, and we get our funding through that. So we will be conservative in some choice that influence the design, which might constrain a few languages in their flexibility and how the Notebook document is separated from the kernel or execution engine. We are aware of the implication, we are aware of the limitation and we are slowly working on that.

We have funding for a (small) Jupyter Conference in the next two years, to gather people that are using Jupyter. I suppose we will do at some point a Call fro talk proposal, I think a "Why I do not use Jupyter with Closure" that have a different view than all the people who are happy and never complain would be good.

I don't know how @aaelony is using Jupyter / closure / Gorilla-REPL, but people that use both project, and are a bit aware of what's happening in each project and/or use both project are crucial in the interoperability and communication between the project. You are part of the people that can come to either Jupyter, or here, and warn us that :

• What we do has already been done.
• what we do will prevent integration with X/Y/Z
• Project W has feature F which is awesome because of ... You should consider it.
• Recap technical choice of project P into something understandable by average developper, and not something like: "The ALU cannot alias 37 bit pointer on the heap because the monad is variadic and the the time complexity limit in O(n^2). It would be better to use algebraic data type" (which is how most cross language discussion sounds like).

So if you have any time to investigate how you feel on both project and make a small writeup, that can be useful.

Long enough gibrish, thanks for pinging me.I have to catch a train unfortunately, so sorry if I don't re-read yself.

[aaelony]

This is a great discussion. Apologies as well (unnecessarily perhaps) for being forced away from this topic until now.

@JonyEpsilon, I agree with (all) your points, and also share your preferences for Clojure and Gorilla. Thanks for following up on this idea/issue. I only meant that it is great that a tool like Jupyter is getting traction and it would be sad if the awesomeness of Gorilla-repl couldn't be included somehow...

@Carreau, I like the idea that the Jupyter team is open to including Gorilla-repl. Perhaps the way forward (?) is to address the points @JonyEpsilon raised by one day in the future providing a gorilla repl option that relied on the gorilla-repl implementation itself, thus allowing different rendering paradigms, immutability, and other differences?

As for myself, my favorite tools are Clojure and R so it would be great to be able to get one experience in a tool like Jupyter. It is nice to see Jupyter and notebooks for multiple languages gaining traction, and it would be great if clojure and gorilla-repl was included. Not sure what the best forum for this idea is, but feel free to close this issue here as necessary. cheers, -A

[joefromct]

FYI, I'm also very interested in this, mostly because my small team enjoys jupyterhub with various kernels and shared notebooks. It's great to showcase an implementation in Python, Scala, Julia, R.... and hopefully someday Clojure.

Clojupyter would be somewhat fitting, however i don't believe it's nearly as mature as Gorilla Repl.... it lacks for instance stack traces and the like. I'd like to use notebooks to showcase the importance of immutable data structures and functional style programming to my team.... for instance with Clojure.

[benfb]

I believe the best thing to do at this point is to use Clojupyter if Clojure + Jupyter functionality is needed, if only due to the sheer amount of effort it would take to combine the two.