LightTable integration

[mmower]

Just looked at #119 where you are thinking about this.

Given that Gorilla sits on top of nrepl and that LT already uses nrepl I wonder how difficult it would be to port the JS parts of Gorilla into an LT plugin.

I know you posted on the LT mailing list about this, I wonder how far you got? Knowing more about Gorilla now as I do I may be in a better position to help (or not...)

[JonyEpsilon]

Yeah, I had a good look at how easy it would be to integrate with LT. And the answer is, unfortunately, not very easy at all!

The problem is that LT is tightly coupled to its editor implementation, so it's not possible, at present, to drop in an alternative editor like Gorilla. It would be quite a lot of work on the LT side, I think, to decouple the IDE part from the editor part.

I don't think I'm in a position to do the decoupling myself, as I found LT's source pretty hard to comprehend - mainly because of the complete absence of documentation, but also because of the rather extreme decentralisation that the BOT design results in. So, I'd say this is unlikely to happen until if/when someone else refactors the LT internals to decouple the editor.

[mmower]

I'm probably missing something Jony but as a plugin you seem to have carte blanche to put whatever you like in your view. Do you need to integrate at the editor level itself? Or just get connected to an nrepl session?

[JonyEpsilon]

Yes, you can do whatever you want with the view, but it you want to hook into the behaviours that an editor should support, like load and save file commands for instance, then you'll find you need to be specifically a LT editor and not any other kind of view :-(

If you look at the command handler for save, for example, you'll see that it directly calls into the editor pool to get the last active editor: https://github.com/LightTable/LightTable/blob/master/src/lt/objs/opener.cljs#L220-L224 . And the editors in the pool have to be LT editors. It would be better if the editor-like behaviours were decoupled so any object could implement editor-like behaviour. I suspect it would probably require a rewrite of the focus/active logic as well.

[JonyEpsilon]

To answer your other question, yes it would be possible to just use the view as a window into Gorilla, and use LT's nREPL connection, but I'm not sure I see this as adding very much over what exists at the minute. It would be a full integration with LT, including things like file management, that would seem appealing.

[mmower]

One of my concerns about LT is that the team have lost interest before the point where it's internals are shored up enough that a community of willing volunteers could realistically take over. The complete lack of documentation was also a big turn-off for me when I was building my first plugin. I soldiered through but have never been back inside.

That said a simple integration: Gorilla in an LT tab, using LT's connection manager, but it's own file management would still be a win for me and might help promote Gorilla.

More people using it might lead to more people asking for better integration and move the LT folks to sort out some of the internals.

Just a thought.

m/

[JonyEpsilon]

Note to self: open a LT issue about decoupling the editor component.

[kurtharriger]

Another possibility perhaps would be an atom plugin (https://atom.io/).

Although I have not tried to build a plugin with it yet the documentation seems pretty good and there are already some clojure and nrepl plugins. If integrating as a plugin to the editor itself is too difficult at the moment you might be able to lift it into atom-shell or node-webkit.

[JonyEpsilon]

I did a bit of experimenting with atom plugins. A problem I ran into is that atom uses its own view framework (spacepen) and I couldn't get knockout.js, which Gorilla uses for its UI updates, to play nicely with it. I know that atom has started using react internally, so there's hope for figuring it out, but in a couple of days of playing, I couldn't get anything to work.

I put it to the atom discussion list, but didn't get anything useful back: https://discuss.atom.io/t/using-knockout-js-in-editor-plugin/10644

So, I wouldn't rule it out, but I think it might be tricky, and it's not at the top of my personal priority list. PR's welcome of course!!

[kurtharriger]

I don't know that its that high of a priority for me either, but I saw this thread and figured I'd mention it. I just found this and seemed like full editor support would be nice, more so for directory and file navigation stuff but it seems to work pretty well in the browser.

I have done some things with React and its not well documented, but pretty easy to create a React "black-hole" that allows you to take ownership of the DOM node and render whatever you want.

module.exports = React.createClass({
  render: function() {
    return 

  },
  componentDidMount: function() {
    var node = this.getDOMNode());
    // this is a raw div node into which you can provide to any other framework attach event listeners and whatever
  }
}

I'm thinking of adding a few features such as auto saving on change and hiding sections for printing. The templates look pretty clean and maybe when I have a little extra time I'll see if I can't pull out the UI and convert it to React.

[kurtharriger]

So I thought I'd spend some hack time this long weekend to see if could get gorilla repl running in atom shell. This Is all still a bit experimental at the moment, but I got it to work on my machine and I thought I would post a update and see what you think about the changes so far. https://github.com/kurtharriger/gorilla-notebook

Rather than copy the UI and diverge completely, I decided to pull the web assets into a new repo and publish them as a node module that can be shared and reused.
https://github.com/kurtharriger/gorilla-web

I added an npm plugin to gorilla repl and the gorilla-web dependency to the resources folder. When gorilla-repl is packaged as a jar it will still include a copy of the web assets and serve them pretty much as it has always done.

I also added CORS headers to gorilla-rep so that the web editor/viewer can be served from any web server (node, atom-shell, aws, whatever). The UI will look for a query string ?server=http://localhost:8990 and connect to that, if not specified it assumes that the web and api are being served from the same server so that gorilla-repl continues to work as it always has. https://github.com/kurtharriger/gorilla-repl

I also started to refactor gorilla-web to use webpack thinking I would add or rewrite some components in React, but haven't really done so yet. Webpack has a pretty amazing hot-reload functionality that allows it to inject and update css/javascript on a running page without even doing a browser refresh (See demo here: https://github.com/gaearon/react-hot-loader) . I'm not loading CSS yet via webpack so that won't hot reload yet, but much of the javascript has been updated to use require for dependencies.

[JonyEpsilon]

Hi Kurt,

sorry for the slow reply, wanted to wait until I had enough time to take a proper look.

I came unstuck in your install script at the grunt task. It doesn't seem to actually download atom-shell for me, and I haven't ever used grunt, so am not immediately sure what's going wrong.

Running gorilla in atom-shell would be cute, but I'm not sure it's a direction that I'd personally see the project going in. The problem that I see, is that if we were to package up gorilla in atom-shell then we'd have to maintain builds of Gorilla for each platform, and my past experience of that hasn't been very positive!

What would, I think, be much more interesting, would be to get Gorilla running inside an already-existing editor. Both because we then wouldn't be responsible for distributing the platform specific stuff, and - more importantly - because we could then integrate with the file/connection management offered by the editor. To me, this last point would be the real appeal of integrating with LightTable or Atom.

Regarding the React stuff, it would be interesting to see how that works out, but I should be upfront that I'd be very unlikely to consider a switch from Knockout to React unless there was a really compelling reason.

Anyway, apologies if that sounds a bit negative - I'm really happy to see people experimenting, but I do want to try and make sure I'm open about the direction I see things going in. And, of course, I'm always happy to entertain arguments that I'm wrong :-)

Jony

[kurtharriger]

Yeah I agree. An atom plugin would be much more useful then atom shell as the main point would be to leverage the features and extensibility of the editor. But I thought I would start with atom shell to see if I could easily separate the UI and the API. I didn't really think about the distribution/platform specific complexities, but that's a good point as well.

Atom shell does allow access to the filesystem to create a more intuitive load/save worksheet to Documents folder or elsewhere. I'm thinking perhaps hosting the api on an internal server, but due to the arbitrary code execution bit it would obviously need to be trusted clients only. Then it becomes much more useful for the client to load and save worksheets locally, although the ability to export and upload worksheets to a database or other user specific storage may work just as well (perhaps datomic since it keeps revision history).

I figured the UI would be more reusable if not so tightly coupled to the API server project. So I pulled that into a separate repo, I'm not sure if you would be interested in separating these into separate repos as I have done since porting changes to/from other clients would be easier, but since I'm not sure if I will actually reuse much of it yet it might be a bit premature. Either way though I think adding CORS headers to the API would be helpful and I can cherry-pick that commit if your interested in a PR for that.

[JonyEpsilon]

Yup I agree back! It certainly would be possible to make a nice "native" app UI with atom shell, but I don't think I'd want to take on the maintenance burden associated with it.

Regarding the decoupling, I think the right thing to do is to wait until there's a need to do so, as the single project has the advantage of simplicity.

I'm not sure I properly understand the CORS idea. That would let you host the web-app on a different domain to the REPL server bridge, right? Is there a particular use case you have in mind for that?

[kurtharriger]

Cross-Origin-Resource-Sharing basically just allows the UI to be hosted by a different origin. A different port on the same server is also considered a different origin and thus the same-origin-policy prevents POST requests to a url other than that which served the web page. Thus if the UI is served via node instead of java / gorilla-repl the same-origin-policy will prevent the browser from making any POST requests to the server.

CORS is something the api server must opt-in to by sending a specific set of headers via an preflight OPTIONS request to determine if the browser should be authorized to POST data to the (api) server. The best guide I've found for understanding how CORS works is here: http://www.html5rocks.com/en/tutorials/cors/

My implementation is here: https://github.com/kurtharriger/gorilla-repl/commit/f395b1cb4b15cbc5033a1094987b2aedf45b363a

[JonyEpsilon]

I think I'm going to close this issue for now, as I don't see the LT integration as a realistic thing right now. Always happy to reconsider this, though, if there are developments with LT that might make it more feasible.