MD cells don't close at end of file

[sho-87]

If you have an existing ipynb file that has a MD cell as the final cell, when you JupyniumLoadFromIpynbTab the ju file contains an MD block at the end that doesn't have a closing tag.

This leads to some errors because it's effectively ending the file on an open """:

[sho-87]

also, if I manually close that MD cell using %%""" to get rid of the error then I'll end up with an empty code cell at the end of the file (due to #50)

[kiyoon]

It's because the %%""" is technically a code cell separator in jupynium file format. For most cases it's faster to type, but some edge cases like this isn't really well-covered.

A better format would be to integrate Jupytext's markdown cell with multi-line string.

# %% [markdown]
"""
This is a markdown cell
"""

The only implementation difficulty is that, the current Jupynium parser only gets partially updated lines, and it is hard to know the context when the formatting requires parsing multiple lines. This is something I didn't consider when I first designed the format.

A workaround for now is to create a closing separator that is not opening a code cell. The end-of-line closure can be something like %%%""" and it will be just ignored by Jupynium parsers.

[sho-87]

create a closing separator that is not opening a code cell. The end-of-line closure can be something like %%%"""

I like that idea as a temporary workaround as it solves some other problems of having double-use separators

[kiyoon]

@sho-87 You might be interested in #68.

When I first made Jupynium, I only cared about Python and made the Jupynium file format.
However, I realised that people have many different workflows and thus it makes more sense to follow the standardised Jupytext format.

From now on, we can deprecate the old Jupynium format and use Jupytext. The benefit is that I don't have to implement conversion tools and I can also deprecate ipynb2jupy command. People can use Jupytext plugin or jupytext CLI to convert. Also this format will not have an edge case like you had with this issue.

[sho-87]

a standardized format sounds like a great idea - using mature libraries to handle parts of the workflow sounds like it'd save a lot of dev time as well. looking forward to this!