Convert Working Mode to Markdown

[annevk]

Fixes #27.

[annevk]

One problem is that "Anchor permanence" now has a new anchor! I guess we can fix this on the whatwg.org side by doing something ugly.

It's also not clear to me if Markdown to HTML will do the right thing with the remainder. I didn't have a convenient way to test locally.

[domenic]

What was the problem with testing locally? I recall being able to set up the two repos side by side and run the scripts in a similar way to how CI does.

[domenic]

I finally worked on testing this. It didn't work out so great; it barfed around the "Additions" section and stopped converting lists and links and code. See https://gist.github.com/domenic/b15343839507835133efffeeb4e408dd

[domenic]

You can fix some of these by closing the </p>s (prevents total breakdown from that point forward), and moving the opening and closing p tags to their own lines (which allows the Markdown processor to work between them on their contents). But that still leaves you with bad interactions with list items. For example

* The feature being removed must either be not widely implemented, or must in the process of being removed from implementations.

*  A test ensuring the feature is not supported must be added and existing tests for the feature should be adjusted or removed as appropriate.
  <p class="example">
  [w3c/web-platform-tests#5001](https://github.com/w3c/web-platform-tests/pull/5001) added new tests that appcache was not supported in shared workers, per the removal in [whatwg/html#2384](https://github.com/whatwg/html/pull/2384).
  </p>
  <p class="example">
  Adjusting existing tests can be difficult. If necessary, an issue can be filed instead to track updating those tests. This was done in [w3c/web-platform-tests#5053](https://github.com/w3c/web-platform-tests/issues/5053) which accompanied the removal in [whatwg/html#2402](https://github.com/whatwg/html/pull/2402).
  </p>

becomes

<ul>
<li>
<p>The feature being removed must either be not widely implemented, or must in the process of being removed from implementations.</p>
</li>
<li>
<p>A test ensuring the feature is not supported must be added and existing tests for the feature should be adjusted or removed as appropriate.
  <p class="example">
  <a href="https://github.com/w3c/web-platform-tests/pull/5001">w3c/web-platform-tests#5001</a> added new tests that appcache was not supported in shared workers, per the removal in <a href="https://github.com/whatwg/html/pull/2384">whatwg/html#2384</a>.
  </p>
  <p class="example">
  Adjusting existing tests can be difficult. If necessary, an issue can be filed instead to track updating those tests. This was done in <a href="https://github.com/w3c/web-platform-tests/issues/5053">w3c/web-platform-tests#5053</a> which accompanied the removal in <a href="https://github.com/whatwg/html/pull/2402">whatwg/html#2402</a>.
  </p></p>
</li>
</ul>

(note the misplaced </p>).

[domenic]

Based on https://spec.commonmark.org/dingus/?text=*%20Test%0A%0A*%20Test%0A%20%20%3Cp%20class%3D%22note%22%3ETest%3C%2Fp%3E%0A%20%20%3Cp%20class%3D%22note%22%3ETest%3C%2Fp%3E it looks like CommonMark could accomodate us and we just have a not-as-good Markdown processor. So, I'm not sure whether swapping that out is feasible, but that'd be one route.

[domenic]

I pushed some work to attempt to fix this, combined with using the commonmark python package, but it didn't work. I can't figure out how to get Markdown to work inside a paragraph with a class name in CommonMark: see https://spec.commonmark.org/dingus/?text=%3Cp%20class%3D%22example%22%3E%0AThe%20%60console%60%20object%0A%3C%2Fp%3E%0A

[domenic]

I guess we could just use https://spec.commonmark.org/dingus/?text=%3Cdiv%20class%3D%22example%22%3E%0A%0AThe%20%60console%60%20object%0A%0A%3C%2Fdiv%3E but it's pretty unsatisfying. The input is weird to read, and the output has a redundant div.

[annevk]

I looked a bit at different Markdown solutions, but I couldn't really find anything satisfying there. So instead I added some syntax and decided it was okay if note and example paragraphs are not part of list items.

So SG policy documents can now start a paragraph with NOTE: or EXAMPLE: and convert-policy.py will take care of turning that into the appropriate class name.

If this is acceptable we can finally move this remaining document to Markdown.

[domenic]

I agree with the idea of using NOTE: or EXAMPLE:. I'm minorly sad about losing the ability to have nested notes or examples. Why doesn't it work to use syntax like

* A test ensuring the feature is not supported must be added and existing tests for the feature should be adjusted or removed as appropriate.

  EXAMPLE: [w3c/web-platform-tests#5001](https://github.com/w3c/web-platform-tests/pull/5001) added new tests that appcache was not supported in shared workers, per the removal in [whatwg/html#2384](https://github.com/whatwg/html/pull/2384).

  EXAMPLE: Adjusting existing tests can be difficult. If necessary, an issue can be filed instead to track updating those tests. This was done in [w3c/web-platform-tests#5053](https://github.com/w3c/web-platform-tests/issues/5053) which accompanied the removal in [whatwg/html#2402](https://github.com/whatwg/html/pull/2402).

? I think it would work in Commonmark...

[othermaciej]

Is there a good way to see the output and compare to the pre-markdown version?

[annevk]

Once https://github.com/whatwg/whatwg.org/pull/295 lands I'll get back to this and will manually include the relevant diffs in comments.

[annevk]

https://github.com/whatwg/whatwg.org/pull/296/commits/155e960b3854736d2a1f3ac06c7469335dc0e995#diff-098172da47da15f436da92f384af214f can be used to compare the output difference.

[foolip]

I've compared the built output visually before/after this and https://github.com/whatwg/whatwg.org/pull/240 and can spot only one difference, which is that the heading level of "New proposals" has changed from <h3> to <h4>.

@annevk was this an intentional change? Either way would work, but the original isn't obviously wrong to me.

[foolip]

@annevk confirmed that wasn't intentional. Fixed and confirm that the before/after look the same now.

I'll merge this and https://github.com/whatwg/whatwg.org/pull/240 at the same time to not have to worry about the right staging order. (The wrong order would delete https://whatwg.org/working-mode)

[foolip]

The deploy worked and https://whatwg.org/working-mode now is now the generated variant.

Note that it's still the case that there's no CI set up for this repo, so just changing files here doesn't cause them to be published. (Merging https://github.com/whatwg/whatwg.org/pull/240 was the trigger this time.)