Dealing with ruleₒₚₜ in the language grammar conversion to markdown

[abelbraaksma]

Not sure if there's been prior discussion on this, but in #4, I proposed here (https://github.com/fsharp/fslang-spec/pull/4#discussion_r1614220206) to not use ~opt in this case, as that has no special rendering. In rules like the following, this would lead to rather confusing syntax:

pats := pat , ... , pat
field-pats := field-pat ; ... ; field-pat
rules := '|'~opt rule '|' ... '|' rule

which was originally rendered as follows:

Or this one:

record-fields :=
    record-field ; ... ; record-field ;~opt

which was rendered like this (i.e., optional trailing ;)

---

Unfortunately, keeping the original text (as I proposed there) isn't too good either. I don't like the rendering in browsers of fixed width fonts when encountering opt in subscript codepoints: testₒₚₜ (notice the pt too close and the o rendered oddly). These are mistakes in the used fonts, the rendering is fine outside of fixed width fonts.

Now, originally I hoped to replace this with a question mark, but the way the current language grammar is written, and with ? being part of the syntax (see A.1.9.2), that's not such a good idea after all.

TLDR: Perhaps for now, unless we find a nice single character (inverted q. mark?), we could go for superscript. It is better supported in browsers. Still not pretty, but better: testᵒᵖᵗ.

The original example then becomes, which is not too bad:

pats := pat , ... , pat
field-pats := field-pat ; ... ; field-pat
rules := '|'ᵒᵖᵗ rule '|' ... '|' rule

Other suggestions? Admittedly, this wasn't a very creative solution :)

[abelbraaksma]

Or maybe just go with the question mark anyway. We should then probably just change the parts of the syntax that currently isn't valid ABNF (like the lists of keywords), but then at least it won't conflict with the existing question marks in the F# syntax.

It is the standard ABNF character for options, after all. The given example would then look like:

pats := pat , ... , pat
field-pats := field-pat ; ... ; field-pat
rules := '|'? rule '|' ... '|' rule

Isn't that clearer than any Unicode sub/sup creativity?

[brianrourkeboll]

I guess you could convert those sections to $\LaTeX$, but that's probably too much work just to get subscripts :)

```math
\begin{equation}
\begin{aligned}
pats              & := pat\ ,\ ...\ ,\ pat \\
field\text{-}pats & := field\text{-}pat\ ;\ ...\ ;\ field\text{-}pat \\
rules             & := |_{opt}\ rule\ |\ ...\ |\ rule \\
\end{aligned}
\end{equation}

```math
\begin{equation}
\begin{aligned}
pats              & := pat\ ,\ ...\ ,\ pat \\
field\text{-}pats & := field\text{-}pat\ ;\ ...\ ;\ field\text{-}pat \\
rules             & := |_{opt}\ rule\ |\ ...\ |\ rule \\
\end{aligned}
\end{equation}

[abelbraaksma]

@brianrourkeboll I didn't even consider that, though it does render beautifully! But... considering the sheer number of examples in the spec, and that we'd like to make it relatively easy to make changes, perhaps $\LaTeX$ is not the best way forward 😆.

EDIT: another downside is that it won't be trivial to copy/paste.

[brianrourkeboll]

@abelbraaksma Yeah, I was hoping some straightforward regex replace would be enough, but probably not. It does read nicely, though.

```math
\begin{equation}
\begin{aligned}
expr & := \\
    & \texttt{const} & \text{a constant value} \\
    & \texttt{( expr )} & \text{block expression} \\
    & \texttt{begin expr end} & \text{block expression} \\
    & \texttt{long-ident-or-op} & \text{lookup expression} \\
    & \texttt{expr . long-ident-or-op} & \text{dot lookup expression} \\
    & \texttt{expr expr} & \text{application expression} \\
    & \texttt{expr( expr )} & \text{high precedence application} \\
    & \texttt{expr< types >} & \text{type application expression} \\
    & \texttt{expr infix-op expr} & \text{infix application expression} \\
    & \texttt{prefix-op expr} & \text{prefix application expression} \\
    & \texttt{expr .[ expr ]} & \text{indexed lookup expression} \\
    & \texttt{expr .[ slice-ranges ]} & \text{slice expression} \\
    & \texttt{expr <- expr} & \text{assignment expression} \\
    & \texttt{expr , ... , expr} & \text{tuple expression} \\
    & \texttt{new type expr} & \text{simple object expression} \\
    & \texttt{\{ new base-call object-members interface-impls\ \}} & \text{object expression} \\
    & \texttt{\{ field-initializers\ \}} & \text{record expression} \\
    & \texttt{\{ expr with field-initializers\ \}} & \text{record cloning expression} \\
    & \texttt{[ expr ; ... ; expr ]} & \text{list expression} \\
    & \texttt{[| expr ; ... ; expr |]} & \text{array expression} \\
    & \texttt{expr \{ comp-or-range-expr\ \}} & \text{computation expression} \\
    & \texttt{[ comp-or-range-expr ]} & \text{computed list expression} \\
    & \texttt{[| comp-or-range-expr |]} & \text{computed array expression} \\
    & \texttt{lazy expr} & \text{delayed expression} \\
    & \texttt{null} & \text{the ``null'' value for a reference type} \\
    & \texttt{expr : type} & \text{type annotation} \\
    & \texttt{expr :> type} & \text{static upcast coercion} \\
    & \texttt{expr :? type} & \text{dynamic type test} \\
    & \texttt{expr :?> type} & \text{dynamic downcast coercion} \\
    & \texttt{upcast expr} & \text{static upcast expression} \\
    & \texttt{downcast expr} & \text{dynamic downcast expression} \\
    & \texttt{let function-defn in expr} & \text{function definition expression} \\
    & \texttt{let value-defn in expr} & \text{value definition expression} \\
    & \texttt{let rec function-or-value-defns in expr} & \text{recursive definition expression} \\
    & \texttt{use ident = expr in expr} & \text{deterministic disposal expression} \\
    & \texttt{fun argument-pats -> expr} & \text{function expression} \\
    & \texttt{function rules} & \text{matching function expression} \\
    & \texttt{expr ; expr} & \text{sequential execution expression} \\
    & \texttt{match expr with rules} & \text{match expression} \\
    & \texttt{try expr with rules} & \text{try/with expression} \\
    & \texttt{try expr finally expr} & \text{try/finally expression} \\
    & \texttt{if expr then expr elif-branches}_{opt}\texttt{ else-branch}_{opt} & \text{conditional expression} \\
    & \texttt{while expr do expr done} & \text{while loop} \\
    & \texttt{for ident = expr to expr do expr done} & \text{simple for loop} \\
\end{aligned}
\end{equation}

```math
\begin{equation}
\begin{aligned}
expr & := \\
    & \texttt{const} & \text{a constant value} \\
    & \texttt{( expr )} & \text{block expression} \\
    & \texttt{begin expr end} & \text{block expression} \\
    & \texttt{long-ident-or-op} & \text{lookup expression} \\
    & \texttt{expr . long-ident-or-op} & \text{dot lookup expression} \\
    & \texttt{expr expr} & \text{application expression} \\
    & \texttt{expr( expr )} & \text{high precedence application} \\
    & \texttt{expr< types >} & \text{type application expression} \\
    & \texttt{expr infix-op expr} & \text{infix application expression} \\
    & \texttt{prefix-op expr} & \text{prefix application expression} \\
    & \texttt{expr .[ expr ]} & \text{indexed lookup expression} \\
    & \texttt{expr .[ slice-ranges ]} & \text{slice expression} \\
    & \texttt{expr <- expr} & \text{assignment expression} \\
    & \texttt{expr , ... , expr} & \text{tuple expression} \\
    & \texttt{new type expr} & \text{simple object expression} \\
    & \texttt{\{ new base-call object-members interface-impls\ \}} & \text{object expression} \\
    & \texttt{\{ field-initializers\ \}} & \text{record expression} \\
    & \texttt{\{ expr with field-initializers\ \}} & \text{record cloning expression} \\
    & \texttt{[ expr ; ... ; expr ]} & \text{list expression} \\
    & \texttt{[| expr ; ... ; expr |]} & \text{array expression} \\
    & \texttt{expr \{ comp-or-range-expr\ \}} & \text{computation expression} \\
    & \texttt{[ comp-or-range-expr ]} & \text{computed list expression} \\
    & \texttt{[| comp-or-range-expr |]} & \text{computed array expression} \\
    & \texttt{lazy expr} & \text{delayed expression} \\
    & \texttt{null} & \text{the ``null'' value for a reference type} \\
    & \texttt{expr : type} & \text{type annotation} \\
    & \texttt{expr :> type} & \text{static upcast coercion} \\
    & \texttt{expr :? type} & \text{dynamic type test} \\
    & \texttt{expr :?> type} & \text{dynamic downcast coercion} \\
    & \texttt{upcast expr} & \text{static upcast expression} \\
    & \texttt{downcast expr} & \text{dynamic downcast expression} \\
    & \texttt{let function-defn in expr} & \text{function definition expression} \\
    & \texttt{let value-defn in expr} & \text{value definition expression} \\
    & \texttt{let rec function-or-value-defns in expr} & \text{recursive definition expression} \\
    & \texttt{use ident = expr in expr} & \text{deterministic disposal expression} \\
    & \texttt{fun argument-pats -> expr} & \text{function expression} \\
    & \texttt{function rules} & \text{matching function expression} \\
    & \texttt{expr ; expr} & \text{sequential execution expression} \\
    & \texttt{match expr with rules} & \text{match expression} \\
    & \texttt{try expr with rules} & \text{try/with expression} \\
    & \texttt{try expr finally expr} & \text{try/finally expression} \\
    & \texttt{if expr then expr elif-branches}_{opt}\texttt{ else-branch}_{opt} & \text{conditional expression} \\
    & \texttt{while expr do expr done} & \text{while loop} \\
    & \texttt{for ident = expr to expr do expr done} & \text{simple for loop} \\
\end{aligned}
\end{equation}

[Martin521]

Yes, I was struggling with the "opt" (see the remark in the conversion guideline). Latex is beautiful, but I really believe we should keep the document format as simple as possible. I like the ?. First, because it is simple and readable. Second, because, over time, I believe we should go to ABNF (or preferably some simpler EBNF dialect) for the whole grammar, so that we can validate it. Else we won't get rid of all the problems it contains now. Finally, making the change should be easy.

[abelbraaksma]

Oh, I missed that in the conversion guide:

We don't support subscripts and just remove the space (expr1). An exception is the "opt" suffix, which we convert into ~opt (e.g. expr~opt), so that we can later easily deal with it differently by using search-and-replace.

I think we actually can support subscripts, especially when numeric, as they are supported in Unicode and in all browsers. But we can deal with them when we get there.

And: in non-code segments, we can use <sub>|<sup> if we so please.

Latex is beautiful, but I really believe we should keep the document format as simple as possible.

Indeed. And you lose the copy/paste possibilities, and search capabilities. Plus, if you do search and find an example in GH, you won't be able to consider it F# in the search results, as that will show $\LaTeX$ code.

First, because it is simple and readable. Second, because, over time, I believe we should go to ABNF (or preferably some simpler EBNF dialect) for the whole grammar, so that we can validate it.

Thanks for supporting the idea to go for ABNF (or EBNF). I agree. There are some good examples of dealing with this type of grammar (i.e. with rule exceptions that cannot be expressed in BNF) in the W3 language grammars. I can help here.

Else we won't get rid of all the problems it contains now.

agreed

Finally, making the change should be easy.

@Martin521: Let's do it! So: question mark it is.

[Martin521]

Good. I will in the coming days make the changes then in all chapters

[Martin521]

I have changed chapters 1 to 6 to the new notation. I will do the remaining chapters when we come closer to reviewing them.

[abelbraaksma]

Awesome. I think we can close this as done.