Experiment with generating an mdBook-based variant of the SRS

[balacij]

Project Proposal

mdBook is used to build the Rust Book, and it looks pretty good to me. For reference, mdBook...

• is heavily Markdown based,
• uses a file-based structure, and
• should support our needs nicely (code [see above example] and math).

This project should be fun. Approximately, the steps should be...

1. Create a demo of the SRS with mdBook. Manually porting over one of the examples.
2. Create a Markdown definition DSL in Drasil.
3. Create an encoding of an mdBook-based project.
4. Create a renderer for the SRS into the mdBook-based project encoding.

Recalling that Markdown is really just a language over HTML, we can (1) reuse our special HTML components in particular. Furthermore, since we have MathJaX support in mdBook, we can (2) reuse our LaTeX renderer.

Project Progress

Development is ongoing by @BilalM04 under the mdBookPrinter and mdBookPrinterTest branches.

TODO:

• [x] Create a demo of the SRS with mdBook. Found here.
• [x] Generate a single-page SRS in Markdown
  • [x] Render the layout objects
  • [x] Header
  • [x] Paragraph
  • [x] Table (raw markdown should be visually readable)
  • [x] Definition Table (raw markdown should be visually readable)
  • [x] List
  • [x] Figure
  • [x] Bib
  • [x] Render the Mathjax expressions
  • [x] Render captions using HTML
  • [x] Render reference anchors
  • [x] Render reference link syntax
• [x] Generate the different SRS sections in separate Markdown files
• [x] Fix reference links to jump across files
• [ ] Figure out how to render assets (Most recent update: assets must be in src folder)
• [x] Generate a SUMMARY.md
• [x] Generate a Book.toml

[smiths]

@balacij I like this project, especially if the finished product looks something like the Rust Book example. I like the table of contents on the left-side pane. If the md document looked very similar to our html version of the SRS, we won't have gained very much. Once it works for the SRS, it would be great if we could also get it to work for the Jupyter notebook work that Ting-Yu did. The mdBook seems like a nice format for presenting material for teaching purposes.

[BilalM04]

@balacij @smiths I am interested in this project. Are there any existing documentation on SRS rendering/generation that may be helpful in getting started. I understand the first step would be to manually create an example; however, going through the Drasil code for the existing SRS formats, I felt a bit lost. I want to gain a general understanding of SRS generation before going into the project. Also, how feasible does this project sound for a beginner in Haskell, but a quick learner?

[balacij]

Not that I know of yet, but I'm working on some through the file I presented in our May 6th meeting (#3715). I wouldn't be too worried about being able to understand the Haskell code right now -- there's nothing too fancy in the Haskell code, it's just large and pieces might be far apart from one another. You'll definitely come to understand it along the way.

To get a better idea of what (4) would entail, you can look at any of the examples' (e.g., projectile's) SRS section declarations, trace where they're used to build a 'document', trace where said document is used in generation, see how the code generator works at a high level, and then slowly dissect how things work. The packages you'll be most interested in are drasil-docLang and drasil-printers. drasil-printers declares a generic document language that can be rendered into TeX, HTML, etc., while drasil-docLang is where the SRS format is defined and exposes a renderer from said format to drasil-printers' generic document language (GDL).

The steps I provided strays away from the high-level code (where do files go, what are the files, what information goes where) of the existing document generation code in Drasil, but tons of the lower-level code (how Exprs translate to TeX, etc.) should be reusable. I'm sure we can figure out a proper flow for the code if you want to take up the project.

I think it's feasible. There's plenty of help around, and, as with all work on Drasil, there will be group activity :slightly_smiling_face:

[smiths]

Great job with the explanation and all the links @balacij. If @BilalM04 is interested in this project, it feels like a nice worthwhile, but feasible, piece of work. I think it would be a candidate for an end-of-summer poster. :smile: A nice thing about md is that it looks good with an html viewer. Unlike pdf files for the requirements, there is no need to download anything. Someone can just navigate to the file and view it. I could see this becoming our "go to" version of the SRS, replacing the html version as the most convenient.

[balacij]

Thanks! I never thought about that (it being a candidate for a poster), but it sounds great! I didn't really think about that either (it becoming the "go to" SRS version), but I can definitely see that happening. With the prospective mdBook variant splitting up the SRS into pages, I think we can get really creative with information presentation. For example, if users had to switch pages to look up an acronym, that could be annoying, so we could mitigate that through two ways: (i) showing long-versions of acronyms on first instance for each page, and (ii) adding tooltips to each instance of an acronym (e.g., NASA <- hover your mouse to see the example). Similarly, with references, we can add on-page listings of relevant sources. I'm sure that extends to other things too, like definitions of SRS-relevant words, descriptions of instance models, etc.. That being said, tooltips can probably be added to the existing HTML SRS variant too, but I only thought about this when I thought of the frustration of having to manually navigate between pages to see relevant information, while in a single webpage, I would just hit CTRL+F and search for whatever I needed.

[samm82]

I (very unprettily) forked the mdBook repo and made a branch (patch-1) with some changes based on my image observations (in case anyone else was curious). This includes using the image URL from our website and adding a "tooltip" which is kind of similar to a native "caption" (and gets added to the HTML as a title attribute that is only displayed upon hovering). This image link seems to be a byproduct of generating and then hosting our website, leading to a bit of a "chicken and the egg" situation if we choose to refer back to this when generating other artifacts; would these types of links be stable enough to reference from within our generated web-based artifacts? Is there a situation where we would generate an example but not host it on our website? It's also possible that this is only an issue since this mdBook version is not generated, since generating it would include adding our images to its datafiles (although this may be a hack).

I also noticed that mdBook's Markdown flavour seems to generate an HTML "stub" for a caption (see the empty <strong></strong> below), but @BilalM04 and I were unable to figure out how to populate this through Markdown syntax (hence the needed for the "manual" HTML added on that same line with the actual caption).

<div id="Figure:sysCtxDiag">
<p><img src="../images/SystemContextFigure.png" alt="System Context Figure">
<strong></strong></p><p align="center"><strong>System Context</strong></p><p></p>
</div>

[BilalM04]

Brief Progress Update

Created a TODO list in the OP of steps that immediately jump out that need to be completed. Checked off the ones I have already completed.

Current Task: Generate a single page Markdown SRS. I opted to begin with generating a single page SRS as it is easier for debugging when rendering different components, and once a single page is nicely printed, separating it into different files should not be too difficult.

Notes from #3760

• For definitions, keep the table format but include headers
• Render the tables to be visually readable in raw markdown
• Clean up the <div> tags
• Fix the white spacing in the raw markdown; maintain consistency

Update

The current state of the generated SRS can be found here.

• Printed regular tables and definition tables to be visually readable
• Cleaned up the white spacing
• Cleaned up the <div> tags to be right above the referenced material, rather than wrapped around
• added headers to definitions table

Immediate Action Items

• Expressions are currently rendered as desired. The Drasil code for rendering expressions uses TeX, where it redefines the functions that are different in mdBook and delegates to the TeX functions otherwise. Investigate other ways to print expressions.
• The bibliography is currently rendered using HTML, it needs to change to pure Markdown.
• Definition table headings design choice - see next comment for question

[BilalM04]

From #3760, it was decided to keep the table format for definitions, but add a heading using the Label field.

The two options for headings were:

1) Heading aligned left with a gray refname aligned right

## Calculation of Landing Time <div style="float: right; color: grey; font-size: 12pt;"><em>IM:calOfLandingTime</em></div>

2) Heading with no refname

## Calculation of landing time {#IM:calOfLandingTime}

---

I opted for option 2 as for the longer headings, the refname and title got very close together and it no longer looked nice:

---

My question is how should the headings be aligned?

Tables in mdbook are automatically centered in the page. For the smaller tables that don't span the entire width, the headings aligned to the left look odd. But for other tables it looks fine.

Aligning headings in the center requires HTML hacks, whereas left aligned can be done in pure markdown (see option 2 above):

<h2 id="TM:velocity" align="center">Velocity</h2>

Additionally, some headings include Mathjax equations, for example:

As a result, the heading tags then need to span across multiple lines for the markdown and mathjax to be rendered:

<h2 id="DD:speedIY" align="center">

\\(y\\)-component of initial velocity

</h2>

[smiths]

@BilalM04 I agree that the centred heading does look nicer. Part of the reason it looks odd for short headings is that the table is centred. If the table was left-justified it might look more natural. If at any point we can't decide how something should look, we could make it a documentation variability and make specifying the formatting value part of the recipe. :smile:

[JacquesCarette]

I agree with @smiths .

[balacij]

Great progress! Thank you for the update!

Generating mdBook does bring up a design issue with the existing printers. Right now, drasil-gen's use of drasil-printers largely expects to generate a single file, and perhaps copying over some assets. I'm not sure if we want to build a miniature virtual file system, but it might be good for us to do when we want to build cross-file references without hardcoding implicit knowledge about how references are formed. I haven't thought this completely through yet, however, so take this comment with a grain of salt.

Also, from the mdBook documentation on its Markdown flavour, it looks like we can do a few neat things:

1. enable smart punctuation (I don't see any downsides to this?),
2. use footnotes (not that we currently do this, but for in-file citation references, having the relevant citations just placed on-page might be nice, in addition to a References page),
3. heading attributes, which I think we can use to replace the HTML use around headings with Markdown and some CSS.