Syntax extensions should be configurable

[chrisjsewell]

As per https://myst-parser.readthedocs.io/en/latest/syntax/optional.html and https://github.com/executablebooks/myst-vs-code#preview-enhancement, all syntax extension should be configurable, to enable/disable. I don't think is possible yet @agoose77?

[agoose77]

It's currently possible to disable jupyterlab-markup extensions, with a checkbox:

These settings can also be set via the disabled-plugins property in the jupterlab-markup settings.

[chrisjsewell]

Ah ok cool 👍

[agoose77]

@chrisjsewell can this be closed?

[chrisjsewell]

yep, although at some point we should write a brief bit of documentation about how to use/configure the extension (in the README or better a small RTD site)

[agoose77]

Sounds good, let's leave this open then.

[tavin]

I think it doesn't work this way anymore? In jupyterlab==3.6.1 and jupyterlab_myst==1.1.1? There seems to be no way to toggle the syntax extensions.

[agoose77]

@tavin good spot; we upgraded the package, and lost some "features" along the way. The extensibility of MyST in general is something that the community at-large need to figure out. Although there is prior art for what this might look like, this fits into the wider context of what version of MyST (e.g. 2023.02.03) is in use, i.e. the need for a version identifier. We also want to make it convenient to extend the syntax of myst at the data-level, so that it can easily be adapted to particular use cases.

So, for now, watch this space.

[tavin]

Good to know @agoose77. Where in the code is the list of enabled extensions, for the sake of hacking on it locally?

[agoose77]

Some of the plugin configuration is here: https://github.com/executablebooks/jupyterlab-myst/blob/cc1ad5a61512c12084b83d61bdcc0ab57f45387b/src/myst.ts#L122-L131

[rowanc1]

And here for the roles/directives:

https://github.com/executablebooks/jupyterlab-myst/blob/main/src/myst.ts#L51-L54

[tavin]

Here's an idea about where to configure syntax extensions:

_config.yml
---
parse:
  myst_enable_extensions:

Maybe for this to work out, a user should somehow actively mark a directory as a "jupyter book project" or simply a "jupyter project" ...

[tavin]

@agoose77 @rowanc1 I have a partial solution for this in #115. Through a labextension "token" I expose a simple interface that encapsulates the options object which is passed to the myst parser.

The currently hardcoded roles, directives, syntax extensions, etc. are kept as defaults.

It's relatively easy to write a labextension to alter these. You can imagine backing the interface with entries from the UI settings editor, or from notebook metadata, or whatever.

I've used this to implement a few custom directives as well.

For context and easy reference these are the options I'm talking about, which can be passed through the interface:

myst-parser/src/types.ts

export type AllOptions = {
  vfile: VFile;
  markdownit: MarkdownIt.Options;
  extensions: {
    colonFences?: boolean;
    frontmatter?: boolean;
    math?: boolean | MathExtensionOptions;
    footnotes?: boolean;
    deflist?: boolean;
    tasklist?: boolean;
    tables?: boolean;
    blocks?: boolean;
  };
  mdast: MdastOptions;
  directives: DirectiveSpec[];
  roles: RoleSpec[];
};

[rowanc1]

This is super cool, still looking through the PR!

Do you have a demo of how to create a different lab extension to override the options?

[tavin]

@rowanc1 this is a fully working labextension which modifies the myst parser options: https://github.com/tavin/myst-options-poc/tree/main/src

[meldefon]

Hello, just wanted to add that this would be extremely helpful! I need a small number of additional directives from sphinx proof for my class materials, and writing would be much easier and cleaner if I could see these directives rendered in the notebook.

[rowanc1]

Hi both @tavin and @meldefon -- I am hoping to get some time next week to come back to focusing on the extension. @agoose77 not sure if you have played with @tavin's proof of concept yet?

[rowanc1]

@meldefon we just added support for sphinx proof here: https://myst-tools.org/docs/mystjs/proofs-and-theorems Let us know what you think. An update should be coming soon to jupyterlab-myst to consume the new updates. Not a solve for this issue, but thought I would mention it since you are using that library!

[meldefon]

This is fantastic!! Thank you very much. The editing experience is massively improved.

Sorry to keep demanding things, but I'm guessing that many of the users who need sphinx proof directives would also benefit from having sphinx exercise. Might be worth incorporating into the upcoming update.

[rowanc1]

@meldefon added support for sphinx-exercise in https://github.com/executablebooks/mystjs/pull/399, this will be coming to jupyterlab soon after a few releases of the theme. 🚀