add rdf/turtle codemirror mode (#326)

[sillymoi]

Discussion in https://github.com/vatlab/sos-notebook/issues/326

[BoPeng]

Are you familiar enough with JS to change the static declaration of

 "codemirror/mode/turtle/turtle",

to something like the following?

load_module("codemirror/mode/turtle/turtle")

I can distribute all codemirror modes with a sos distribution, but apparently it is a bad idea to load them all when the sos kernel loads.

[sillymoi]

something like require? do you currently do any dynamic loading like that? also, does this need to be added to sosActionWords?

[BoPeng]

I think sos-notebook uses require but I do not know how to use it and this is why I used static dependency. If you know how to use require to load the codemirror mode dynamically, I can implement other pieces of the puzzle, namely, moving the mode-loading logics to a user-editable configuration file.

Otherwise we can wait till next month when a programmer will join my team, who is supposed to know JS well.

[sillymoi]

codemirror has an example with dynamic loading (https://codemirror.net/demo/loadmode.html), the bit of code is in change():

CodeMirror.modeURL = "../mode/%N/%N.js";
var editor = CodeMirror.fromTextArea(document.getElementById("code"), {
  lineNumbers: true
});
var modeInput = document.getElementById("mode");
CodeMirror.on(modeInput, "keypress", function(e) {
  if (e.keyCode == 13) change();
});
function change() {
  var val = modeInput.value, m, mode, spec;
  if (m = /.+\.([^.]+)$/.exec(val)) {
    var info = CodeMirror.findModeByExtension(m[1]);
    if (info) {
      mode = info.mode;
      spec = info.mime;
    }
  } else if (/\//.test(val)) {
    var info = CodeMirror.findModeByMIME(val);
    if (info) {
      mode = info.mode;
      spec = val;
    }
  } else {
    mode = spec = val;
  }
  if (mode) {
    editor.setOption("mode", spec);
    CodeMirror.autoLoadMode(editor, mode);
    document.getElementById("modeinfo").textContent = spec;
  } else {
    alert("Could not find a mode corresponding to " + val);
  }
}

[sillymoi]

do you have any pointers where to start? i'm ok with js, but kernel.js itself is huge 😄

[sillymoi]

p.s. i see at the very end that you had some problems with autoLoadMode() (issue #55)?

[BoPeng]

kernel.js itself is huge 😄

It only shows that I did not even know how to split the function into multiple js files.

kernel.js gets a list of kernels their mode using winow.CodeMirrorMode from

https://github.com/vatlab/sos-notebook/blob/032b74d7c7ff6a9a9b667e995b635d9383ded6b1/src/sos_notebook/kernel.js#L447-L457

So it is possible that, to add

    CodeMirror.autoLoadMode(editor, mode);

after cell.code_mirror.setOption (line 457), and

CodeMirror.modeURL = "../mode/%N/%N.js";

somewhere in the front.

Then we can try to remove the long list of modes

https://github.com/vatlab/sos-notebook/blob/032b74d7c7ff6a9a9b667e995b635d9383ded6b1/src/sos_notebook/kernel.js#L7-L27

and see if the autoLoadMode works.

The read config parts is at

https://github.com/vatlab/sos-notebook/blob/032b74d7c7ff6a9a9b667e995b635d9383ded6b1/src/sos_notebook/kernel.js#L1940-L1953

which should be easily expanded to allow users to specify a map between kernel name and codemirror mode name....

[BoPeng]

A complication is that

cell.user_highlight 

is a sos mode that works with a bunch of base_mode to add sos-specific highlighting. For testing purposes we can use base_mode directly and ignore the sos part for now.

[sillymoi]

found the bit with window.CodeMirrorMode, what was confusing is that when i try debugging this in the browser (on jupyter-lab), there is no such thing on window.. guess my problem is that i've seen jupyter/sos innards a few days ago :)

i'm getting a lot of No base mode is found for turtle. Python mode used., this line is commented out though.. am i running this thing wrong? did python setup.py install in sos-notebook and started jupyter-lab

[BoPeng]

@CharlesZhou58 in case you can help.

If you are using jupyterlab, then this part is handled in jupyterlab-sos .... so this PR should not work at all.

[sillymoi]

🤦 ok, let me check with the right code

[sillymoi]

when i make the similar changes to codemirror-sos.ts in jupyterlab-sos, i get the highlighting for turtle, so no problem there. my situation is a bit odd since i have multiple kernels using the same language (turtle), i narrowed down to my entry point configuration for sos plugin:

entry_points= '''
[sos_languages]
turtle1 = mykernel.kernel:sos_MyKernel
turtle2 = mykernel.kernel:sos_MyKernel
turtle3 = mykernel.kernel:sos_MyKernel
'''

is there any way to override the highlighting? at https://github.com/vatlab/jupyterlab-sos/blob/master/src/codemirror-sos.ts#L261 there is no base_mode passed in parserConf, but it does work if i don't add a sos extension myself.

[BoPeng]

is there any way to override the highlighting?

@CharlesZhou58 and I will be working on this in the next few weeks. The goal is to allow users to specify codemirror mode for kernels. We will focus on sos-notebook before we move to jupyterlab-sos later.

A bit more details:

1. The configuration file is ~/.jupyter/nbconfig/notebook.json. It will be written by python -m sos_notebook.install if it does not exist.
2. In this file, we could do something like this

{
    'languages': {
        'turtle1': {
            'kernel': 'mykernel.kernel:sos_MyKernel',
            'codemirror_mode': 'turtle',
            'background': 'red',
        }
    }
}

3. Then, in kernel.js, this configuration file will be loaded and supplement what sos-notebook detects from the system. The codemirror mode will be dynamically loaded.

4. If this works well, all statically loaded codemirror mode will be included in this configuration file and allow users to customize.

In this way no language module would be needed for kernels that do not really to exchange data with sos, but need their own codemirror modes.

[CharlesZhou58]

Hi @sillymoi I have downloaded the project to my computer and installed jupyter, sos-notebook, jupyterlab-sos. Could you please help me how to run this project and reproduce the problem you have? Thank you very much.

[BoPeng]

@CharlesZhou58 You will need to install the turtle kernel from https://github.com/sillymoi/turtle-kernel . Then the challenge is how to use the turtle mode of codemirror to highlight the cells that use the turtle kernel.

Currently, sos-notebook only supports a pre-defined, statically loaded code-mirror modes, and it makes more sense to use a configuration file to direct how to load additional kernels in cases like this.

[BoPeng]

Fixed.

[sillymoi]

Fixed.

sorry for the late reply, checked and it is looking good. thank you!