tooling, MD guidelines

[Martin521]

This can perhaps be a place to share what we learn while converting the spec.

I have been looking into how C# does it. Some cool stuff:

• Automatic extration of examples (for verification and testing)
• Automatic extraction of the grammar (so the grammar annex is created automatically)
• ANTLR grammars (that is probably not a viable short-term goal for us)
• Header generation and references

Most of this is probably not for now for us.

But while looking into it I learned that we should quote code not by indented code blocks (as I did in the first versions of my chapters), but by fenced code blocks with a fsharp info string like this

let x = 1

to get syntax coloring for free.

I also think we should add an info string to the fence of each grammar block. I propose we use "fsgrammar" for now. This will enable us to later on to programmatically find / change those grammar elements. See chapter 3 for examples.

[edgarfgp]

Yeah once we have all the content chapters we should look into some automation alternatives

[Martin521]

Manual conversion effort that is needed:

• Add indentations to code examples and to syntax
• Add fsharp or fsgrammar info string to code block fences for code and syntax, resp.
• Turn all blue items in text paragraphs (blue color in the pdf spec) into code spans (i.e. enclose them in back ticks). If they are in italics, enclose the code span by underlines.
• Reconstruct tables (using GFM (github flavoured markdown))
• Check for occasional problems in the auto-converted version (for example missing spaces around dashes)
• Add spaces back to table like syntax definition (like in the operator lists in section 4.1)
• Adjust heading levels (# 1. / ## 1.1 / ### 1.1.1)
• Revert headings back from all caps to normal capitalization (CONSTANTS -> Constants)

Open problems

• In the pdf spec, in the syntax boxes, grammar elements are shown in italics. I propose to ignore this for now, as there is no simple markdown means to do it.
• Same for subscripts / superscripts

[edgarfgp]

Yes I started doing some manual conversion in 5.1 will continue doing that next week

[Martin521]

I added a build script which collects all chapters, updates section headers, and creates the full spec with index.

Please rename the chapter 5 file to `types-and-type-constraints.md" and add it toclauses.json`. Let me know if you come across problems with the build script.

[edgarfgp]

I added a build script which collects all chapters, updates section headers, and creates the full spec with index.

Please rename the chapter 5 file to `types-and-type-constraints.md" and add it toclauses.json`. Let me know if you come across problems with the build script.

Renamed the file and updated the clauses.json but got an exception when running the build script.

[Martin521]

Which exception?

[edgarfgp]

System.InvalidOperationException: The input list was empty.
   at Microsoft.FSharp.Collections.FSharpList`1.get_Head() in D:\a\_work\1\s\src\FSharp.Core\prim-types.fs:line 4142
   at FSI_0001.renumberIfHeaderLine[a](a clauseName, BuildState state, String line) in /Users/edgar/code/fsharplang/build.fsx:line 65
   at FSI_0001.renumberClause@75.Invoke(BuildState state, String line)
   at Microsoft.FSharp.Primitives.Basics.List.mapFoldToFreshConsTail[TResult,TState,T](FSharpList`1 cons, FSharpFunc`3 f, TState acc, FSharpList`1 xs) in D:\a\_work\1\s\src\FSharp.Core\local.fs:line 375
   at Microsoft.FSharp.Primitives.Basics.List.mapFold[TState,T,TResult](FSharpFunc`2 f, TState acc, FSharpList`1 xs) in D:\a\_work\1\s\src\FSharp.Core\local.fs:line 393
   at FSI_0001.renumberClause(BuildState state, Clause clause) in /Users/edgar/code/fsharplang/build.fsx:line 75
   at FSI_0001.build@80.Invoke(BuildState state, Clause clause)
   at Microsoft.FSharp.Primitives.Basics.List.mapFold[TState,T,TResult](FSharpFunc`2 f, TState acc, FSharpList`1 xs) in D:\a\_work\1\s\src\FSharp.Core\local.fs:line 386
   at FSI_0001.build() in /Users/edgar/code/fsharplang/build.fsx:line 80
   at <StartupCode$FSI_0001>.$FSI_0001.main@() in /Users/edgar/code/fsharplang/build.fsx:line 89
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
   at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)

[Martin521]

Thanks. The script found a header inconsistency and reported it, but then also produced an exception. The latter is fixed now and you will see the reported errors. For example ### 5.1 Checking Syntactic Types should be ## 5.1 Checking Syntactic Types. (The header levels must not increment by more than 1.)

[edgarfgp]

Thanks. It is working now

[Martin521]

So, another thing solved that people thought "is pain" (toc generation) :-)

[edgarfgp]

For notes should we use ?

[!NOTE] Im a note

[edgarfgp]

Also §5.4.7 we could change this to (See 5.4.7) ?

[Martin521]

Agree to using > for notes. Not sure what you refer to for §5.4.7

[edgarfgp]

I used > markdown

> [!NOTE]

[Martin521]

Agree to using > for notes. Not sure what you refer to for §5.4.7?

[edgarfgp]

Agree to using > for notes. Not sure what you refer to for §5.4.7?

I mean once we have all the chapter converted we can change all (See 5.4.7) to be md links

[Martin521]

Ah, ok. Yes, I have planned already how to do that, similar to cslang, with automation.

[Martin521]

Ah, ok. Yes, I have planned already how to do that, similar to cslang, with automation. Better to keep the § in for now, so the tool can find all the references.

[Martin521]

Regarding notes: it seems that [!NOTE] is a github extension of standard markdown. It is not displayed, for example, in VS Code. I also checked and found that csharpstandard does not use it. I therefore suggest that we use > **Note**: This is the note text .... The result is also visually not too different from the original.

[Martin521]

I added tooling for creation and maintenance of the reference links.

[Martin521]

Pages 68 .. 70 of the pdf spec (part of §6.3.10) look like copied and pasted from a scientific paper, introducing all sorts of Greek and math symbols. Almost impossible to get this into markdown.

[edgarfgp]

@Martin521 Just finished chapter 16. We have only left:

• 18(FSharp.Core). Few content. mostly broken links that we can replace with https://fsharp.github.io/fsharp-core-docs/reference/fsharp-core.html ?
• 19(ML Compatibility)
• Appendix A: F# Grammar Summary.

Should we convert these too ?

[Martin521]

I was just starting 18. Yes, we should do 18 and 19 because there are some references to them in earlier chapters. Grammar summary I am not sure. In principle, this should only be an extraction of all the grammars from the main text. Personally, I would rather look into automation or more formal grammars before starting that. Also glossary and index can be done later and need some automation that I would be happy to do.