Add support for markdown statements.

[jsannemo]

This PR adds rudimentary support for markdown statements, together with some extra markdown statements for example problems (and a scoring example problem).

Supported:

• samples, including interactive
• tables
• inline math mode using \( \)
• problem2html

Planned to be added when I add problems that use the feature:

• footnotes
• images

Not supported:

• problem2pdf
• everything else

This is completely untested in combination with Kattis problem installation, since that's a pain to test locally for me.

[RagnarGrootKoerkamp]

Nice! We've been considering to add this to the spec as well.

I think it's important to spec this well. Some things that come to mind:

• I'd prefer to also allow the usual $ delimiters.

• We should spec whether the first line must be the problem title or not. Latex has \problemname{NAME} on the first line, so I think it makes sense to have # NAME as the first line of a markdown statement.

  • Alternatively, we could render the title from the name in the problem.yaml automatically.

• Similarly, statements should have \subsection{Input} or the equivalent \begin{Input} ... \end{Input} (which is nice because you can't accidentally mistype Input or put \section instead of \subsection). Some form of checking may be nice. (Or encourage using the latex commands directly, but it's ugly.)

• I think Pandoc (which I've been trying in BAPCtools for md->tex->pdf rendering) would still support things like $\newcommand$, which is nice to extract common bounds. Does this implementation also do this?

• For images, one way would be to do it directly via the latex syntax: $\illustration{0.3}{image.jpg}{CC-BY}$. (Ah, but you're probably doing the md->html rendering directly, so maybe it's better to somehow parse the standard markdown image inclusion separately for html/pdf rendering. Sounds non-trivial though.)

[jsannemo]

Nice! We've been considering to add this to the spec as well.

I think it's important to spec this well. Some things that come to mind:

• I'd prefer to also allow the usual $ delimiters.

I always felt that syntax in MD to be slightly iffy. In LaTeX escaping $ is part of the syntax, in markdown it is not. You'd have to resort to heuristics to detect math instead, which feels much harder to spec properly.

• We should spec whether the first line must be the problem title or not. Latex has \problemname{NAME} on the first line, so I think it makes sense to have # NAME as the first line of a markdown statement.

This PR was based on the spec, which suggests that the name entry in problem.yaml should be used instead (and I agree with the spec).

• Alternatively, we could render the title from the name in the problem.yaml automatically.

That is what I assume the --title option to problem2html does, but I didn't actually check that verifyproblem passes it

• Similarly, statements should have \subsection{Input} or the equivalent \begin{Input} ... \end{Input} (which is nice because you can't accidentally mistype Input or put \section instead of \subsection). Some form of checking may be nice.

Except for interactive problems, or just problems in general where communication is done in other ways (such as IOI-style with e.g. language specific wrappers). But either way, this feels like something tp be discussed in a new issue.

• I think Pandoc (which I've been trying in BAPCtools for md->tex->pdf rendering) would still support things like $\newcommand$, which is nice to extract common bounds. Does this implementation also do this?

No

• For images, one way would be to do it directly via the latex syntax: $\illustration{0.3}{image.jpg}{CC-BY}$. (Ah, but you're probably doing the md->html rendering directly, so maybe it's better to somehow parse the standard markdown image inclusion separately for html/pdf rendering. Sounds non-trivial though.)

The plan is to add a post-processor for img tags that fixes links to images pointing to an image file.

[RagnarGrootKoerkamp]

Nice! We've been considering to add this to the spec as well. I think it's important to spec this well. Some things that come to mind:

• I'd prefer to also allow the usual $ delimiters.

I always felt that syntax in MD to be slightly iffy. In LaTeX escaping $ is part of the syntax, in markdown it is not. You'd have to resort to heuristics to detect math instead, which feels much harder to spec properly.

Hmmm yeah - I'm a bit torn on this. $ is supported by pandoc which is the de facto standard for math in markdown, but probably doesn't have a nice clean spec. I think not allowing $ (and thus parsing $ as normal text) is going to be ugly as well.

Also came across this, which actually argues against \( because that's an escaped parenthesis in plain markdown.

On this topic, we also have to choose a base markdown spec - probably commonmark is the most universal one (as opposed to e.g. github flavored markdown).

• We should spec whether the first line must be the problem title or not. Latex has \problemname{NAME} on the first line, so I think it makes sense to have # NAME as the first line of a markdown statement.

This PR was based on the spec, which suggests that the name entry in problem.yaml should be used instead (and I agree with the spec).

Hmm where exactly did you read this? This is all tricky since officially the name field in problem.yaml isn't part of the ICPC spec... Although it is what I'm doing for latex as well. IMO this entire problem name part of the spec could use some cleaning up. Actually I'd prefer to only put the name in the problem.yaml and not again as the first line of the statement, as you currently propose, but the asymmetry with latex statements is annoying.

• Alternatively, we could render the title from the name in the problem.yaml automatically.

That is what I assume the --title option to problem2html does, but I didn't actually check that verifyproblem passes it

• Similarly, statements should have \subsection{Input} or the equivalent \begin{Input} ... \end{Input} (which is nice because you can't accidentally mistype Input or put \section instead of \subsection). Some form of checking may be nice.

Except for interactive problems, or just problems in general where communication is done in other ways (such as IOI-style with e.g. language specific wrappers). But either way, this feels like something tp be discussed in a new issue.

See #180 ;) But yeah, it's definitely out of scope to add commands/environments for all possible use cases. But in both BAPC and NWERC we had some potential bugs where people used the wrong type of section heading so I'd like to somehow guard against this.

• I think Pandoc (which I've been trying in BAPCtools for md->tex->pdf rendering) would still support things like $\newcommand$, which is nice to extract common bounds. Does this implementation also do this?

No

Ack. Actually I'd be ok with having slightly less latex capabilities in a markdown file - if needed you can always revert to the equivalent latex. This means we'd have to make sure that the layout for e.g. lists is exactly equivalent between the two.

• For images, one way would be to do it directly via the latex syntax: $\illustration{0.3}{image.jpg}{CC-BY}$. (Ah, but you're probably doing the md->html rendering directly, so maybe it's better to somehow parse the standard markdown image inclusion separately for html/pdf rendering. Sounds non-trivial though.)

The plan is to add a post-processor for img tags that fixes links to images pointing to an image file.

Makes sense.

CC @niemela, who has also been wanting/pushing this.

[RagnarGrootKoerkamp]

For reference, GitLab uses:

$`abc`$

for inline maths (which I don't like so much) and

```math
abc

for display math, which is quite nice actually, although I'm not sure whether pandoc does this.

Actually, if we're doing maths, we should definitely support both inline and display maths.

[jsannemo]

On Thu, 18 Mar 2021, 01:05 Ragnar Groot Koerkamp, @.***> wrote:

Nice! We've been considering to add this to the spec as well. I think it's important to spec this well. Some things that come to mind:

• I'd prefer to also allow the usual $ delimiters.

I always felt that syntax in MD to be slightly iffy. In LaTeX escaping $ is part of the syntax, in markdown it is not. You'd have to resort to heuristics to detect math instead, which feels much harder to spec properly.

Hmmm yeah - I'm a bit torn on this. $ is supported by pandoc which is the de facto standard for math in markdown, but probably doesn't have a nice clean spec. I think not allowing $ (and thus parsing $ as normal text) is going to be ugly as well..Also came across this https://talk.commonmark.org/t/ignore-latex-like-math-mode-or-parse-it/1926/3, which actually argues against ( because that's an escaped parenthesis in plain markdown.

That is true (the Kattis implementation parses is configured to parse math and ( before escapes for this reason).

I am not married to ( at all though, and am happy to use another sufficiently easy-to-distinguish and simple-to-type delimiter.

On this topic, we also have to choose a base markdown spec - probably commonmark is the most universal one (as opposed to e.g. github flavored markdown).

• We should spec whether the first line must be the problem title or not. Latex has \problemname{NAME} on the first line, so I think it makes sense to have # NAME as the first line of a markdown statement.

This PR was based on the spec, which suggests that the name entry in problem.yaml should be used instead (and I agree with the spec).

Hmm where exactly did you read this? This is all tricky since officially the name field in problem.yaml isn't part of the ICPC spec... Although it is what I'm doing for latex as well. IMO this entire problem name part of the spec could use some cleaning up. Actually I'd prefer to only put the name in the problem.yaml and not again as the first line of the statement, as you currently propose, but the asymmetry with latex statements is annoying.

I thought LaTeX statements can use the name mechanism already today.

• Alternatively, we could render the title from the name in the problem.yaml automatically.

That is what I assume the --title option to problem2html does, but I didn't actually check that verifyproblem passes it

• Similarly, statements should have \subsection{Input} or the equivalent \begin{Input} ... \end{Input} (which is nice because you can't accidentally mistype Input or put \section instead of \subsection). Some form of checking may be nice.

Except for interactive problems, or just problems in general where communication is done in other ways (such as IOI-style with e.g. language specific wrappers). But either way, this feels like something tp be discussed in a new issue.

See #180 https://github.com/Kattis/problemtools/pull/180 ;) But yeah, it's definitely out of scope to add commands/environments for all possible use cases. But in both BAPC and NWERC we had some potential bugs where people used the wrong type of section heading so I'd like to somehow guard against this.

• I think Pandoc (which I've been trying in BAPCtools for md->tex->pdf rendering) would still support things like $\newcommand$, which is nice to extract common bounds. Does this implementation also do this?

No

Ack. Actually I'd be ok with having slightly less latex capabilities in a markdown file - if needed you can always revert to the equivalent latex. This means we'd have to make sure that the layout for e.g. lists is exactly equivalent between the two.

For me, markdown is convenient when I churn out problems or assist people who do not know LaTeX in preparing simple contests. That it lacks all the options of LaTeX is IMO fine.

• For images, one way would be to do it directly via the latex syntax: $\illustration{0.3}{image.jpg}{CC-BY}$. (Ah, but you're probably doing the md->html rendering directly, so maybe it's better to somehow parse the standard markdown image inclusion separately for html/pdf rendering. Sounds non-trivial though.)

The plan is to add a post-processor for img tags that fixes links to images pointing to an image file.

Makes sense.

CC @niemela https://github.com/niemela, who has also been wanting/pushing this.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Kattis/problemtools/pull/191#issuecomment-801520071, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAHIMXOAPDS5CSDA5UJLMK3TEE7V7ANCNFSM4ZLB3UGQ .

[jsannemo]

On Thu, 18 Mar 2021, 01:11 Ragnar Groot Koerkamp, @.***> wrote:

For reference, GitLab uses:

$abc$

for inline maths (which I don't like so much)

The advantage is that it is simpler to preprocess for extensionless markdown implementations since you can care less about avoiding math delimiters in `` blocks (something which ( also suffers from).

andd

  abc

for display math, which is quite nice actually, although I'm not sure whether pandoc does this.

Actually, if we're doing maths, we should definitely support both inline and display maths.

Definitely. I started with inline since it felt like a smaller syntax to
decide on. I like ```math too.

 —
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<https://github.com/Kattis/problemtools/pull/191#issuecomment-801521734>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAHIMXOXDGTO7UJPW5ARCW3TEFAJRANCNFSM4ZLB3UGQ>
.

[pehrsoderman]

@jsannemo Any progress here?

[pehrsoderman]

@jsannemo Any progress? Also, see comments above...