Automatically generate the Markdown episodes from the Jupyter Notebooks?

[jhlegarreta]

The maintenance burden to keep all contents synchronized across episode Markdown files, Jupyter notebooks and the solution Jupyter notebooks is considerable. Whenever a change is made to an episode, inadvertently leaving behind one of the three parts is quite likely.

It may be worthwhile to explore how feasible is to use Jupytext to automatically (i.e. using the command line interface) convert the notebooks into Markdown, and whether the layout/formatting looks as nice as when writing natively in Markdown. But that might come with some other challenges and might require some work:

• The Jekyll commands (e.g. layout, includes, etc.) might need to be added manually to the generated files, or find some way to nicely (automatically) add what is needed.
• The Carpentries Markdown style recommend line breaks at 100 characters to keep a consistent style and improve readability. Not sure if that can be automated or enforced in case the Markdown files got generated automatically.

[kaitj]

Definitely an option to explore, especially if we want to continue to keep all 3. One thing to keep in mind is that solutions should be removed from the Jupyter notebooks, but retained in the solutions Jupyter notebook - having trouble envisioning a way to easily replace solutions to a more "fill-in-the-blank" method that was being used in fMRI and in previous versions of dMRI. I can see maybe completely removing the content within a coding cell, though not sure if that is an ideal solution.

The Carpentries Markdown style recommend line breaks at 100 characters to keep a consistent style and improve readability. Not sure if that can be automated or enforced in case the Markdown files got generated automatically.

I think we can enforce this using something like markdownlint? Also if this needs to be enforced on the Python side, can use a python linter like black, though I am not sure if this works for notebooks.

[jhlegarreta]

A note on the linters: the Python linter and Markdown linter will probably format the code differently, beyond some basic criteria, like the line length, so if we are successful in using them, we'll need to take into account that the Markdown vs. Python formatting will be different. But automatization on formatting is a highly desirable feature, so I think discrepancy is tolerable.

[jhlegarreta]

Also if this needs to be enforced on the Python side, can use a python linter like black, though I am not sure if this works for notebooks.

Looks like there are a number of tools for notebooks: e.g. jblack , black-nb, jupyter-black