support running a vs-code server in a CoCalc project

[williamstein]

Just like

+New --> Jupyter Classic Server, and +New --> JupyterLab server 

are 1-click ways to work with a CoCalc project through the Jupyter class and JupyterLab interfaces, we should also have a 1-click way to work with a CoCalc through VS Code. This is technically possible, and probably pretty easy, due to the open source MIT licensed code-server project. Here's an example of how that might feel:

https://github.com/betatim/vscode-binder/tree/master

• How To Set Up the code-server Cloud IDE Platform on Ubuntu 18.04 or maybe this
• maybe this

NOTE that evidently a possible shortcoming of code-server is Microsoft's licensing for extensions, which "prohibits use with any non Microsoft product." Since probably any interesting use of CoCalc + VS Code would involve extensions -- e.g., LEAN or Jupyter/Python -- this might be a big problem. Or maybe not. I don't know.

[williamstein]

Same idea but for https://theia-ide.org/ also makes sense. In fact theia may be way better eventually since it is more open source, and uses phosphorjs.... https://news.ycombinator.com/item?id=22738607

[oschulz]

A point in favour of VS-code, compared to Theia, may be it's very broad (and I think still rapidly expanding) user base. Also, VS-code has some awesome extensions for language, with language-server, etc. (the Julia support, for example, it truly first-rate), I don't know how much there is for Theia.

And then there's stuff like the awesome (relatively recent) SandDance extension.

Maybe using Coder's extension marketplace would be enough to deal with the MS VS Code Marketplace licensing?

[oschulz]

Speaking of SandDance, maybe I should open a separate issue for that, just in case you guys aren't busy enough already (making CoCal more awesome) ;-) : #4554

[slel]

Regarding opensourceness, consider also vscodium:

• https://vscodium.com/

VSCodium is a community-driven, freely-licensed binary distribution of Microsoft’s editor VSCode

[oschulz]

@slel, from what I understand, the problem is less with the license of VS-Code itself (code-server is MIT licensed, like the VS-Code source, and I assume CoCalc would use it's own build, not third-party binaries), but with the license under which the MS VS Code Marketplace operates.

Partially, the restrictions of the Marketplace may also be mitigated by the fact that a user can download the extensions from the Marketplace and install the VSIX file in (I guess also a non-MS-built) VS-Code manually. Though maybe the Coder marketplace will do just fine.

[oschulz]

This may be of interest as well: https://github.com/VSCodium/vscodium/blob/master/DOCS.md#extensions-marketplace

[williamstein]

I just spent some time looking into this. code-server definitely looks a lot less "mature" right now than the official Jupyter notebook server (say), which is very very mature, high quality and well documented. In particular, to work cleanly and efficiently in cocalc, the key thing we need from code-server is base url or base path support.

This base url support is the same thing that TONS of other people need in many situations for code-server. You can start to trace the total mess of confusion and opening/closing/reverting/adding/removing bits of this from code-server here:

https://github.com/cdr/code-server/issues/241

To get a full extent of this, you have to dive into closed PR's, etc. In any case, suffice to say that it seems to me that as of now they completely given up on base url support and deleted all the relevant half-way code.

Thus definitely step 1 of code-server integration in cocalc would be getting code-server to entirely build from source in a cocalc project and at least server on localhost qith no base url support. This is super important because step 2 would then be to somehow trace through deleted code from PR's, etc., and somehow implement base url support ourselves (despite the code-server people getting all confused and failing), and maintain that going forward, since clearly they don't think it is worth doing. My guess is that it wouldn't be too hard to hack on the source code to get this to work...

For reference, if you look at the Jupyter notebook server, they have excellent and mature and well documented base url support. They are a shining example of how to do this well.

Oh, an entirely different approach might be to use nginx and mangle all http requests back and forth, but I think this is generally going to be complicated and flakie. It's what you would have to do if you had no choice but to treat code-server as a black box...

A third approach would be to add wildcard url support and a subdomain per project to cocalc. That's a huge can of worms itself, but is what some other sites do. I'm against giving up and using this approach, since it would be very limiting overall (e.g., leaving cocalc-docker out of luck, etc.).

In any case, the answer to

"Maybe CoCalc could spawn that as a service, like it currently allows the user to start JupyterLab and Jupyter-Notebook server instances?"

is no, we can't do it in in the same way, because Jupyter notebook server has key functionality that code-server doesn't.

[oschulz]

they completely given up on base url support and deleted all the relevant half-way code [...]

Uh, yes, I understand why that's a big blocker. Well, maybe in time ...

[williamstein]

For the record, I don't think base url is necessarily an easy problem to solve...

On Sun, May 3, 2020, 11:01 AM Oliver Schulz notifications@github.com wrote:

they completely given up on base url support and deleted all the relevant half-way code [...]

Uh, yes, I understand why that's a big blocker. Well, maybe in time ...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sagemathinc/cocalc/issues/4479#issuecomment-623153441, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJXS5W2IHF3L74X46N6J33RPWWOFANCNFSM4LW6SACQ .

[oschulz]

Indeed - in Jupyter this is of course all well handled, I guess it was also necessary for JupyterHub, etc.

[williamstein]

This is relevant -- https://github.com/cdr/code-server/issues/1739#issuecomment-635431712

[williamstein]

https://news.ycombinator.com/item?id=24705599

[oschulz]

Oh, VS code on Colab, that's cool!

[williamstein]

Yeah -- This now "just works" with no need for basePath's, since they made their url's all relative.

To run this in your own project right now:

1. Install a release (see https://github.com/cdr/code-server/releases)

   wget https://github.com/cdr/code-server/releases/download/v3.11.0/code-server-3.11.0-linux-amd64.tar.gz \
   && tar xvf  code-server-3.11.0-linux-amd64.tar.gz \
   && cd code-server-3.11.0-linux-amd64 \
   && echo "https://cocalc.com/$COCALC_PROJECT_ID/server/8000/" \
   && ./bin/code-server --bind-addr=0.0.0.0:8000 --auth=none 

2. Visit this URL in your browser: https://cocalc.com/[project_id]/server/8000/, where obviously you must replace [project_id] by your project's id.

That's it!

I plan to have this preinstalled (and updated regularly) and make starting it a 1-click operation. Also, we can add realtime collaboration as explained here https://github.com/cdr/code-server/issues/33#issuecomment-890184201

[williamstein]

Here's what it looks like in an iframe in cocalc (no collab yet);

To do this, I created a new html file and put this content in it:

<iframe src="https://cocalc.com/10f0e544-313c-4efe-8718-2142ac97ad11/server/8000/" style="width: 100%; height: 95vh"/>

then fullscreeened the right panel showing the content.

[williamstein]

Reminder about extensions. They do install just fine if you have network access. So we need to make it very clear in the UI that you need network access to install extensions.

[williamstein]

Another natural idea would be to write a VScode plugin that saves TimeTravel snapshots of files as they are being edited to our normal TimeTravel history. That way you don't loose out on one of the very nice features of cocalc; you could even imagine having one person edit a file in vscode and another in cocalc's normal editor, and have them sync up (longterm goal).

[williamstein]

Will be live in a few minutes, but to use it, you still have to install code-server as follows:

cd /tmp \
&& wget https://github.com/cdr/code-server/releases/download/v3.11.0/code-server-3.11.0-linux-amd64.tar.gz \
&& tar xvf  code-server-3.11.0-linux-amd64.tar.gz \
&& rm code-server-3.11.0-linux-amd64.tar.gz \
&& mkdir -p ~/bin \
&& cd ~/bin \
&& ln -s /tmp/code-server-3.11.0-linux-amd64/code-server .

Then click +New --> VS Code Server.

[oschulz]

Awesome!

[williamstein]

This is now live and should work in any project. Just refresh your browser, and also for this week you have to select the "Experimental" image in project settings under "Software Environment".

It turns out if you have a file open in both vscode and cocalc, things tend to work "surprisingly" well, since vscode is very aggressive about saving to disk, and cocalc works well with saves to disk.

[oschulz]

Now let's all vote for cdr/code-server#33 ... that would be a very juicy cherry on top ...

[oschulz]

Or is this part if it already (just saw your comment on it, @williamstein )?

[williamstein]

Realtime collaboration is definitely not implemented yet, though I have some ideas. I probably have to write a cocalc-specific VS Code plugin, since I want integration of whatever sync there is for code-server to be compatible with cocalc. I.e., maybe one person is editing a.c in vscode, and another person in cocalc at the same time, and it properly works. That would be cool.

William

On Tue, Aug 3, 2021 at 12:33 AM Oliver Schulz @.***> wrote:

Or is this part if it already (just saw your comment on it, @williamstein https://github.com/williamstein )?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/sagemathinc/cocalc/issues/4479#issuecomment-891609190, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJXS5XCAU2W24EAETFTN4TT26LTDANCNFSM4LW6SACQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

-- -- William Stein

[oschulz]

Several users can use VS-Code server on the same set of files though, as long as they don't edit the same file without reloading, right?

[williamstein]

Several users can use VS-Code server on the same set of files though, as long as they don't edit the same file without reloading, right?

Definitely YES. Even if they are using the exact same files, as long as they don't edit at the exact same time, it still works extremely well. In fact, even if one person is using cocalc and the person is using vscode, as long as the person using cocalc saves to disk frequently, things work pretty well. Basically any time anything saves to disk, code-server reloads that file from disk. Also, code-server very frequently saves to disk itself, and cocalc notices that and loads from disk whenever it happens.

[williamstein]

Also, this is now in the default software environment on both cocalc.com and cocalc-docker, so fully supported by us.