Better support for markdown and LaTeX

[clnsmth]

Currently, EAL converts markdown to TextType sections and paras via read_txt(). Since EDI and DataONE now render markdown in their respective data portals we can now begin writing TextType markdown.

EDI can render LaTeX along side markdown but it cannot be within markdown tags (see methods section of example rendering, and example EML). Does DataONE handle LaTeX + markdown similarly?

Consider an implementation that solves ropensci/EML issue #298.

[servilla]

It doesn't appear that DataONE supports the rendering of LaTex equations: https://search.dataone.org/view/https%3A%2F%2Fpasta.lternet.edu%2Fpackage%2Fmetadata%2Feml%2Fknb-lter-sbc%2F103%2F3

[clnsmth]

Thanks @servilla.

The implementation should take this into consideration

[clnsmth]

There is a solution to mixing LaTeX equations and markdown formatted text within EML nodes, but the EML R library, which EMLassemblyline uses to convert the R list representation to .xml can't handle it. The idea was to parse an .md file's LaTeX into and everything else into so we'd end up with a list of something like:

methodStep = list(
  description = list(
    markdown = md_section_1,
    para = equation_1,
    markdown = md_section_2,
    para = equation_2
  )
)

But EML::eml_write() returns:

<methodStep>
  <description>
    <markdown>md_section_1</markdown>
    <para>equation_1</para>
    <markdown.1>me_section_2</markdown.1>
    <para.1>equation_2</para.1>
  </description>
<methodStep>

Editing the EML file to drop the ".1" may be the hack to get this to work.

[earnaud]

Shouldn't this be adressed as an issue in the ropensci EML package git?

[clnsmth]

Good point @earnaud. I'll open an issue there.

[clnsmth]

The "hack" mentioned above has been implemented. LaTeX equations can now be imbedded in methods.md.

Test drive this feature from the development branch.

[clnsmth]

_NOTE: LaTeX equations must be wrapped in "$$" (i.e. "$$$$")._

[mobb]

Currently, EAL converts markdown to TextType sections and paras via read_txt(). Since EDI and DataONE now render markdown in their respective data portals we can now begin writing TextType markdown.

EDI can render LaTeX along side markdown but it cannot be within markdown tags (see methods section of example rendering, and example EML). Does DataONE handle LaTeX + markdown similarly?

Consider an implementation that solves ropensci/EML issue #298

To the best of my knowledge, two things:

1. DataONE does not render LaTeX.
2. the $$equation$$ construct is used by the renderer, there are probably other ways to trigger a transform. So the LaTeX solution for EAL is limited to EDI, and doesn't apply to the EML-R package in general.

[amoeba]

Hey all, just thought I'd pipe in since a few things caught my eye:

1. That EML serialization thing looks like a bug for sure. I think we may already have an issue filed for it since it looks likes a general issue that'll appear anywhere elements repeat like that. Feel free to file a new issue if you can't find an existing one.
2. DataONE should be rendering LaTeX style equations when they appear in a markdown element so let us know if you're not seeing that. See https://eml.ecoinformatics.org/whats-new-in-eml-2-2-0.html#markdown-support. Support for equations isn't specifically stated in the EML schema (which targets just GFM) but, in practice, we're using http://www.showdownjs.com and intend to make use of Markdown extensions such as equations, citations, etc.

Happy to help answer any questions here or on your Slack.

[clnsmth]

Thanks @amoeba!

RE:

1. I'll open an rOpenSci/EML issue for this serialization thing and see if I have a contribution for https://github.com/ropensci/EML/issues/298 (once I get a few things off my plate).
2. Awesome! Can you point me to an example DataONE data package landing page demonstrating the LaTeX in a markdown element rendering feature? We're using a blend of showdown and MathJax in EDI and may have some questions for how you're getting this to work with showdown alone.

[amoeba]

I'm not sure we have any/many real-world usages yet but I just tossed up a basic example on one of our test sites: https://test.arcticdata.io/view/urn%3Auuid%3A235b3315-0f31-42e7-9fc2-26e817a394a8. I think the magic on our end is initializing Showdown with and these extensions: https://github.com/NCEAS/metacatui/blob/b8c3fb444501843a3564b508604508c634681a7e/src/js/views/MarkdownView.js#L208-L210. Let me know if that helps!

[clnsmth]

Super helpful @amoeba! Thanks!

[clnsmth]

This feature has been merged into the master branch