Convert LaTeX to markdown

[Achllle]

The LaTeX documentation isn't very contributor-friendly and isn't rendered by GitHub so switching to markdown would be better. This would mean quite a bit of effort, but can be broken up into chunks. Once the markdown is generated, we can set up hooks to convert those files into PDFs for convenient printing.

[apollokit]

For converting from markdown to pdf, pandoc looks pretty good, though you might have to tinker with it.

• install pandoc from deb here, to get the latest: https://github.com/jgm/pandoc/releases/tag/2.10
• pandoc README.md -t html5 -o out.pdf
  • Have to install the .deb from https://wkhtmltopdf.org/downloads.html first

[apollokit]

On setting up hooks...

There's some useful info about git commit hooks here: https://www.atlassian.com/git/tutorials/git-hooks

I was thinking that would be a good way to go to automatically produce the pdf docs from markdown upon commit (probably with a pre-commit hook), but I think it's overkill for now. The problem: you want to look at the pdf-rendered-from-markdown before you commit it, to make sure that the doc still looks good given whatever changes you've made. I don't think hooks can be used completely before you start a commit. At that point, all you're really doing is running a bash script anyway, it's not really a "hook" anymore.

I do think, though, that if we're treating the markdown as the Truth document, we could have a hook that checks to make sure there's a corresponding update to any .pdf whose .md has changed - just to make sure the pdfs do get updated (manually).

Probably best just to start with making the markdown, and then worry about automation later.

[SConaway]

I'm willing to help take some of it (any section works for me). Should we try to divide the work?

[capsulecorplab]

Has the team looked into asciidoc? It's a markup language specifically designed for technical writing and documentation. It supports LaTeX and can also output to PDF with asciidoctor , using the asciidoctor-pdf extension. GitHub also renders GitHub flavored asciidoc. We're currently using asciidoctor for maintaining documentation in the form of a static site that also generates pdf documents as part of its CI build on the SoCal Linux Expo A/V repo.

[Achllle]

Thanks for the input!

@SConaway we will start dividing up the work soon, thank you for offering to help!

@capsulecorplab the main reason for switching to markdown is to make the repo contributor and especially beginning contributor friendly. There are several builders who are using git and Linux for the first time. Would you say that asciidoctor fits that description?

[SConaway]

I'd say that asciidoc would be a good fit, especially as it works with LaTeX.

[apollokit]

@capsulecorplab Cool, I didn't know that was a thing. Hopefully it generates better looking pdfs than pandoc (at least, the default output from pandoc).

I don't want to pour cold water on it, but it would be good to know the pros/cons vs github-flavored markdown. The biggest benefit of markdown would be that it's pretty common and people don't have to adopt a new tool. Is there much of the documentation that we're afraid wouldn't render well with markdown?

Note that at this point I'm less gungho on the need to output pdfs automatically (could always just print the github webpage to pdf in chrome)

[capsulecorplab]

If you're writing documentation excusively for the GitHub UI, then the choice between GitHub flavored markdown (GFM) vs. GitHub flavored asciidoc (GFA) is for the most part a matter of personal preference. In terms of pure markup languages, Asciidoc has been argued to have a lighter syntax and consistency compared to markdown (see Asciidoc vs. Markdown). If it's within the roadmap to produce a static site and/or generate other doc formats (s.a., PDF, DocBook, EPUB), then asciidoc(tor) is perhaps the better suited option with its rich set of features and extensions. Asciidoc is also supported on several static site generators.

Hope this helps.

[Achllle]

So far I'm not fully sold on this. I've switched to a different markup language in the past Achllle/dual_quaternions: reStructuredText and found it to be a true PITA to have to refer to documentation the whole time. It looks like AsciiDoc is much closer to MarkDown when it comes to basic functionality though.

Let's take a look at what's in the LaTeX docs right now aside from the most basic markup functionality:

• tables
• images
• images in tables
• side-by-side images

And what's missing in the generated PDFs that we would like:

• internal links and hyperlinks don't work in GitHub
• links to other documents within the repo or other way to easily navigate between pages

Let me know if I'm missing anything. AFAIK GFM can do all of those things.

To reiterate the main concerns:

1. documentation has to be up-to-date
2. documentation has to be complete, detailed, and beginner-friendly
3. documentation has to be contributor-friendly

We hit 2) already because of the great work of @ejunkins. We don't hit 1) but we're slowly chipping away at that and the hope is that that will get accelerated when we hit 3). I know this is contradicting the original issue, but to hit 1), we shouldn't be pushing builders to print the PDFs (we can't really keep those up-to-date 😉), so that would remove the need to convert to PDF.

So if PDF generation becomes less of a concern, the question is: Is GFM good enough to warrant decent navigation between pages or does asciidoc work better for that since we have the ability to generate a static website? With the criteria above, I'd be tempted to say that GFM is probably good enough.

Other considerations:

• once we switch to GFM, moving to asciidoc doesn't seem too much of a stretch. The bigger move is LaTeX -> GFM/GFA
• GFM has much more support: plenty of questions asked on StackOverflow, extensions for every IDE out there, things like a markdown table generator.
• note that the main concerns don't require us to use GitHub/git. We could consider readme.io or community editable wikis.

This is a typical tooling conversation and engineers love to go all in on their favorite tool. I think either solution would be good. "Who merges the PR first makes the decision". So if you're reading this and convinced we should go one way, please open a PR! 🥇

[SConaway]

To aid in the process, pandoc can be used for the initial migration, with humans checking and editing. It supports exporting LaTeX to Markdown or AsciiDoc.

Links to other pages work just like regular links: [title](folder/page.md)

[apollokit]

Sorry, went dark for lil bit.

Did a test to reassure myself on things. I'm honestly not as familiar with GFM as I should be :sweat_smile: . But I'm pretty happy with the image capabilities in GFM now.

Honestly GFM seems fine for the needs of the documentation in these repos. I feel like it'll maximize approachability and ease of keeping up to date.

Note that I count publishing a static site as a non-requirement here. But if that does become one in the future, then agreed, it wouldn't be that bad of a transition from GFM to something else. It's getting it in GFM in the first place that's big deal.

Also, @Achllle , nice, https://www.tablesgenerator.com/markdown_tables is a great tool!

[apollokit]

Note I've been working on updating the README in osr-rover-code, but I'm up for taking a build doc in the main repo too.

[Achllle]

Work has officially started on the conversion process!

The first piece to be converted will be the Wheel Assembly instructions as they need to be overhauled anyways for the upcoming update to the drive motor assembly. You can follow progress on this branch. I expect to be done in a few weeks at most (not a trivial conversion, trying to do two things at once here)

Instead of then converting markdown to PDF, it would be much easier to recommend folks to an existing 'save/print' extension, such as this one for example in chrome which lets you pretty print README pages.

Once that PR is well underway or merged, we will start the process of dividing up the remaining work. We will use project 1 to track progress.

[apollokit]

Anybody landing here, definitely check out the project page before grabbing any of this work. We're Command and Conquering over there, not here.

That said, we can keep this issue open for general discussion.

[apollokit]

@SConaway I'd like to assign https://github.com/nasa-jpl/open-source-rover/issues/212 to you, but for some reason I'm not able to find your username in the assignees drop down. Not sure if I need to add you to the repo somehow. @Achllle are you able to assign him?

[SConaway]

Hmmm. I might need to be in the project for that. If you don't want to do that, you always could put something like (assigned to SConaway).

[ericjunkins]

I tried to assign him to it as well, it wouldn't let me @ him or add to assign i dont know why...

[Achllle]

You can only assign issues to issue commenters (source). So @SConaway simply comment on it and one of the maintainers will be able to assign it to you.

[SConaway]

@Achllle done there.

Besides the SW Controls, what else should I do?

[apollokit]

@SConaway can you comment on https://github.com/nasa-jpl/open-source-rover/issues/217 too, so I can assign you? Probably enough for now.

This is annoying, github.

[apollokit]

Btw @SConaway, definitely join the new slack: https://www.tapatalk.com/groups/jpl_opensource_rover/let-s-give-slack-a-try-t224.html

[apollokit]

Also: made a new channel in the slack, "docs-overhaul" for discussion. Let's try keeping the chatter there instead of here.

[Achllle]

Closing this issue since we've moved efforts to Project 1 and discussion to Slack.