A quick note on the myst spec

[choldgraf]

Thanks for taking a pass at this. A few quick thoughts I noticed when looking at some of the cells, regarding the myst spec:

• way to designate a cell is code-cell, not % cell and % endcell
• You can put front-matter yaml at the beginning of a cell by wrapping it in ---, you don't need a separate ```{metadata} for this
• You also don't need to put {metadata} in the front-matter YAML of the file, MyST assumes that the first block wrapped in --- is YAML for notebook-level metadata

• You can nest markdown literal blocks (which is what MyST uses for cells) by making the outer-most ones of a longer length than the inner-most ones. So you could do:

  ````{code-cell}
  ---
  key: val
  ---
  print("hi")
  
  ```{output}
  hi

The documentation for the MyST-Notebook format is here:

https://myst-nb.readthedocs.io/en/latest/use/markdown.html

though I'm happy to improve that if there are things that are confusing. In addition, there is already a MyST markdown parser here: https://myst-parser.readthedocs.io/ which also describes the syntax a bit more. MyST Notebooks are a subset of MyST Markdown. There's also a Python API for a parser in case that helps with iteration: https://myst-parser.readthedocs.io/en/latest/using/use_api.html

also note: in case it's helpful, we've already got a MyST markdown plugin for vscode, in case it helps figure out the syntax. It'll do highlighting and some completion for you: https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight

I should also note: For obvious reasons I'd prefer not to have two different "MyST-based notebook formats" out there. If there's a way that this approach can be ~ the same as what is in the MyST-NB spec, that would be great. If there are ways that the MyST-NB spec should evolve (e.g. to include some notion of an output block) then we can discuss.

[issue-label-bot[bot]]

Issue Label Bot is not confident enough to auto-label this issue. See dashboard for more details.

[choldgraf]

Another thought here - I think that using the MyST syntax will also result in a pretty big performance hit because it's running regexes for a bunch of things that aren't needed in a MyST Notebooks converter. For example, MyST searches for in-line syntax like {role}`content` as well as markdown syntax like **highlights**. When we are scoping to just converting between ipynb and MyST notebooks, we can basically reduce the tokens that MyST needs to:

```{directive}

to capture the boundaries of cells

and

---

cell: metadata

to capture the notebook-level and cell-level metadata, and maybe the output bundles as well

and that's it. We don't need to care about parsing the markdown **content** of notebooks, because we'd assume that some renderer would do that, not an I/O package. 

Restricting the parsing syntax to just those tokens could speed up reading by quite a bit.