Notes from first use of the CLI

[consideRatio]

These are notes from converting the documentation of the https://github.com/jupyterhub/zero-to-jupyterhub-k8s project. I just wrote that came to mind along as I used it for the first time with no prior knowledge other than its a cli to help me transform rst to myst.

• [ ] roles list command output line separated roles instead of a big string for better human readability.
• [ ] roles show help string sais the same as list.
• [ ] directives list/show same as above

• [ ] --help could perhaps say something more relevant than CLI for rst-to-myst, such as the focus of its purpose.

  Usage: rst2myst [OPTIONS] COMMAND [ARGS]...
  
  CLI for rst-to-myst

• [ ] parse --help or the help of the parse --conversion flag could perhaps provide some reference or be a bit more verbal about it maybe.
• [x] parse --extensions help text doesn't describe how to specify the extensions, comma separated list to load?
• [x] `parse --sphinx/--no-sphinx doesn't make it clear what is the default

• [x] Trailing blank lines.

  • [x] The conversion of a .rst file with only hello world followed by a blank line leads to a .md output of hello world with 3 blank lines
  • [x] The conversion of the following

    .. image:: ../_static/images/architecture.png
    :align: center

    Led to a blank new line in the image directive

    
    ```{image} ../_static/images/architecture.png
    :align: center

• [ ] I noted that my generated myst directives was using colon-syntax for directives like note, and important, but not for code-block. I ended up with a mix of colon-syntax and backtick syntax for my myst directives. I had not enabled colon-syntax manually in any way or by configuration.
• [ ] It would be nice to have a --validate flag or similar for a parsing, giving hints about the transform rather than just outputting the output without warnings etc.
• [ ] I found myself using rst2myst parse -f myfile.rst > myfile.md a lot, it could be nice to have some flag to modify the file in place or similar perhaps. -i, --in-place for example.
• [ ] I learned that it wasn't possible to do rst2myst parse -f test.md > test.md for some reason I don't get. I would guess this would be reasonable to have working in general though, for example cat test.md | sed replacesomething > test.md. Perhaps its because we started printing content before we had finished reading it?
• [x] Having used some double ticks in rst translated to doubleticks in md where I expected to have only one

• [x] I'm surprised to see this conversion of links...

  # rst
  `Google <https://cloud.google.com/>`_)
  
  # myst - why is there a <> remaining? It seems fine though.
  [Google](<https://cloud.google.com/>)

• [ ] I failed to transform the glossary directive to myst, but the automated generated MyST content containing a eval-rst block worked.
• [x] Converting a .. toctree:: directive with a :titlesonly: argument worked in a way, but the generated MyST content was invalid. It became :titlesonly: None which made a difference to the toctree where nothing was shown instead of only the titles.

[welcome[bot]]

Thanks for opening your first issue here! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out EBP's Code of Conduct. Also, please try to follow the issue template as it helps other community members to contribute more effectively.
If your issue is a feature request, others may react to it, to raise its prominence (see Feature Voting).
Welcome to the EBP community! :tada:

[chrisjsewell]

Thanks @consideRatio

I noted that ::: as a directive signal was used an directive that wasn't supported it seems, but was used for a directive that was. That made me a bit confused.

Can you expand on this a bit further, what directives are you referring to?

[consideRatio]

@chrisjsewell I updated the issue! I'm still adding to it btw, just didn't dare keep it in browser-memory without pressing send in between as a save mechanism :p

I noted that my generated myst directives was using colon-syntax for directives like note, and important, but not for code-block. I ended up with a mix of colon-syntax and backtick syntax for my myst directives. I had not enabled colon-syntax manually in any way or by configuration.

[chrisjsewell]

I found myself using rst2myst parse -f myfile.rst > myfile.md a lot, it could be nice to have some flag to modify the file in place or similar perhaps. -i, --in-place for example.

related to #6

[chrisjsewell]

Ha yeh no problem

[chrisjsewell]

I noted that my generated myst directives was using colon-syntax for directives like note, and important, but not for code-block. I ended up with a mix of colon-syntax and backtick syntax for my myst directives. I had not enabled colon-syntax manually in any way or by configuration.

As talked about at https://github.com/executablebooks/rst-to-myst#advanced-usage, the conversion of directives is dictated by https://github.com/executablebooks/rst-to-myst/blob/main/rst_to_myst/data/directives.yml. You can "update" this configuration with the --conversion my_config.yml, or I could add a flag to ignore the colon prefixes.

The idea though is that for directives that contain content that itself is Markdown, it is more helpful in standard renderers (like here on GitHub or viewing Jupyter Notebooks) to see:

:::{note}

This contains Markdown

• With things like links

:::

rather than

> This contains *Markdown*
- With things like [links](https://example.com)

for code-blocks, the content is not Markdown, so then we want ```

In fact, while I'm thinking it now, it would be ideal to have a special case for code/code-block directives, with no options, to convert them to standard language fences, i.e.

.. code-block:: python

   a = "text"

to:

```python
a = "text"

rather than

````markdown
```{code-block} python
a = "text"

[chrisjsewell]

myst - why is there a <> remaining? It seems fine though.

See https://github.com/executablebooks/mdformat/pull/38#issuecomment-699636729, i.e. its "safer", but could be removed with some extra code to check for unsafe links: https://github.com/executablebooks/mdformat/commit/a3b4de32f9fe84ae29c099c1f53262671dbe27c3

[chrisjsewell]

I learned that it wasn't possible to do rst2myst parse -f test.md > test.md for some reason I don't get. I would guess this would be reasonable to have working in general though, for example cat test.md | sed replacesomething > test.md. Perhaps its because we started printing content before we had finished reading it?

Probably to do with the use of https://click.palletsprojects.com/en/7.x/api/#click.File, i.e. the file is still open

[chrisjsewell]

Converting a .. toctree:: directive with a :titlesonly: argument worked in a way, but the generated MyST content was invalid. It became :titlesonly: None which made a difference to the toctree where nothing was shown instead of only the titles

as of v0.1.2, flags (i.e. options with no value) are converted to :key: true which should now work correctly

[chrisjsewell]

parse --extensions help text doesn't describe how to specify the extensions, comma separated list to load?

no at present it would be parse -e ext1 -e ext2 -e ext3, but could do comma separated and also want to add #7

[chrisjsewell]

A lot of this is now fixed in v0.2.0, see https://rst-to-myst.readthedocs.io/

whenever you have time, if you want to check it out and note any outstanding issues

[cjolowicz]

@consideRatio

I learned that it wasn't possible to do rst2myst parse -f test.md > test.md for some reason I don't get. I would guess this would be reasonable to have working in general though, for example cat test.md | sed replacesomething > test.md. Perhaps its because we started printing content before we had finished reading it?

This is a common shell gotcha. The shell truncates the output file when opening for writing, before the command has a chance to open it for reading.