Old vs. new guideline system

[th-we]

I'm certain there are good reasons for maintaining the guidelines as Markdown for Github pages instead of TEI, but those reasons are not apparent to me at the moment and I must confess, I just got frustrated with understanding what's going on while trying to create an admittedly quick pull request on a really late Sunday evening. At the same time I feel really bad about this because I got upset with something I know you guys are investing a lot of time and thought. So please, can you enlighten me?

Here's my rant about things I got frustrated with. If those things are just preliminary because the transition is still work in progress, I apologize - please ignore them in that case. I already anticipate being much less cranky tomorrow morning, but then I'll be occupied with other stuff and this feedback wouldn't be written at all. So enjoy:

Cryptic noise

Now we have

{% include specDesc.html version=page.version elem="itemList" atts="" %}
{% include specDesc.html version=page.version elem="componentGrp" atts="" %}
{% include specDesc.html version=page.version elem="relationList" atts="" %}

where we previously had

<specList>
  <specDesc key="itemList"/>
  <specDesc key="componentGrp"/>
  <specDesc key="relationList"/>
</specList>

And we have

[source]({{ site.baseurl }}/{{ page.version }}/elements/source.html){:.link_odd_elementSpec}

where we used to have

<gi scheme="MEI">source</gi>

oXygen used to help us very nicely with the XML noise - what's going to help us with the awkward boilerplate noise we have to add for all these new things now? I fear guideline writers will just not link to elements and attributes anymore because it's just too much hassle and - even worse - makes the text utterly unreadable. Honestly, who can read something like this?

The content model of the [source]({{ site.baseurl }}/{{ page.version }}/elements/source.html){:.link_odd_elementSpec} element is similar to that of the [fileDesc]({{ site.baseurl }}/{{ page.version }}/elements/fileDesc.html){:.link_odd_elementSpec} and [work]({{ site.baseurl }}/{{ page.version }}/elements/work.html){:.link_odd_elementSpec} elements. The list below reflects the order in which the optional components of [source]({{ site.baseurl }}/{{ page.version }}/elements/source.html){:.link_odd_elementSpec} must occur.

Technically, that's a signal to noise ratio of 0.5. We used to have a ratio of 1.9:

<p>The content model of the <gi scheme="MEI">source</gi> element is similar to that of the <gi scheme="MEI">fileDesc</gi> and <gi scheme="MEI">work</gi> elements. The list below reflects the order in which the optional components of <gi scheme="MEI">source</gi> must occur.</p>

How do we make sure we didn't make a typo anywhere in that boilerplate jungle? And how will this be maintained if there needs to be a change in the boilerplate code that's repeated hundreds of times?

I mean, the reason why markdown exists is because it's simpler and less noisy than HTML. Where a Markdown based solution makes things more complicated than with XML - why do we use Markdown instead of XML in the first place? If it's because it's beneficial to move to Github pages, wouldn't it be possible to generate those from TEI?

Examples in separate files

Examples are an essential part of the guidelines and closely tied to the text. 20% of our examples are one-liners. Can't we keep those together? When editing the guidelines, we must be able to read them in a sensible form.

At least for me, it's harder to make sense of the documentation in markdown form when I don't see the examples (or have to open them one-by-one when reading through while editing).

Linking and TOC related stuff

It seems that chapters need to be numbered manually. Not so convenient, but what's worse, do we have to state the chapter titles (and potentially numbers) explictly everywhere we link to them - and whenever a title changes, we have to change every link caption if we don't want to become inconsistent? Also, this is once more an extreme case of boilerplate overkill:

<ptr target="#headerFileDescription"/>

vs.

<a class="link_ptr" title="File Description" href="{{ site.baseurl }}/{{ page.version }}/guidelines/header.html#headerFileDescription">2.1 File Description</a>

Separation of schema and guidelines

This is just a minor point, but while I'm in full ranting flow: I feel that it makes sense to discuss and perform changes to schema and guidelines in synch. Therefore I don't really see where it helps to have separate repos and issue trackers for them.

Wrap-up

I got the impression, writing guidelines is made much harder rather than simpler. This is unfortunate because it's clear we need more and better documentation. People who are already involved in writing documentation will achieve less if it's made harder. And new editors will more likely be repelled rather than encouraged. All potential guideline editors feel at home with XML and oXygen. Why force them into this new system?

Can you explain the motivation? Is what I criticize going to change?

[lpugin]

What you wrote here is by far more than all the contributions we received to the guidelines over the last two years ;-)

In short, because I will also be occupied with other stuff today: there is an open issue (#7) simplified templates. That is, replace

{% include specDesc.html version=page.version elem="itemList" atts="" %}

with something like

{% include desc elem="itemList" %}

---

And replace

[source]({{ site.baseurl }}/{{ page.version }}/elements/source.html){:.link_odd_elementSpec}

with

{% include link element="mei" %}

---

Or

<a class="link_ptr" title="File Description" href="{{ site.baseurl }}/{{ page.version }}/guidelines/header.html#headerFileDescription">2.1 File Description</a>

with

{% include link target="headerFileDescription" %}

[th-we]

Good morning from the anticipated less cranky version of myself. Thanks Laurent, good to hear the first point is being addressed. Does this also mean any pull requests should wait or be made against the old TEI guidelines because they'll be converted again anyway?

[lpugin]

Please wait for making PR because the content will indeed be overwritten as the migration is still in progress

[ahankinson]

Hi Thomas,

There were several factors that led to the decision to move away from TEI for the Guidelines.

1. The stylesheet for publishing the HTML and PDFs was being hand-maintained by Johannes, and needed extensive "hand-tweaking" after running it to make the content presentable. It also meant that publishing updates to the guidelines was a long and drawn-out process.

2. The Wordpress site that we currently have has proven difficult to maintain, so we were looking for a solution to more easily distribute updates and workloads to members of the community. We decided to move the main website to GitHub pages, since the process for updating the site is fairly well known to many, and easy-to-understand for new people. Laurent uses GitHub pages for quite a bit of the RISM-CH website, and has said that the staff there find it easy to use. Since we were moving the main website, it also made sense to at least investigate the use of GH pages for the Guidelines.

3. There were several levels of indirection that could lead to problems. If we wanted to accept updates to the Guidelines, we asked people to edit a TEI file that would then (at some point; see point 1 above) be transformed into HTML, and then at some point deployed to the website. With GitHub pages, editing a file (once it's been accepted through review) leads to an immediate update on the website. We're hoping that this immediate result of contributions helps community members feel like they're contributing something meaningful.

4. Integrating Verovio into the publishing workflows TEI Guidelines for rendering examples was going to be a tricky task. Thanks to Craig Sapp's efforts in documenting his Verovio-to-Humdrum viewer we have a nice pattern to follow for publishing the Guidelines with renderings of the examples.

In the end we decided that the benefits of keeping all of MEI in a single format were outweighed by the cost of maintaining them in a format that nobody seemed to contribute to, on a server that we needed to maintain ourselves, that led to significant delays in publishing them, and that would restrict our ability to provide useful live rendered examples.

We know there are going to be some growing pains as we make the transition. If you would like to help with some updates I can offer some suggestions.

[th-we]

I appreciate and admire your patience with my impatience, Andrew! Of course that all make a lot of sense and I expect we'll end up in a state where the markdown becomes readable and contributable.

What work suggestions do you have? Are they covered by existing issues or is it something else? I don't know a thing about Github pages, but I can learn. Or maybe I can win over someone who wants to learn about Github pages anyway.

[lpugin]

@th-we, for http://music-encoding.org/guidelines/dev/content/cmn.html#cmnNotesStems you can now compare https://github.com/music-encoding/music-encoding/blob/develop/source/guidelines/cmn.xml#L393-L481 with https://github.com/music-encoding/guidelines/blob/master/_guidelines-dev/04-cmn/01-cmnbasics/04-cmnnoteschords/01-cmnnotes/03-cmnnotesstems.md

[lpugin]

Or for http://music-encoding.org/guidelines/dev/content/cmn.html#cmnRests compare https://github.com/music-encoding/music-encoding/blob/develop/source/guidelines/cmn.xml#L482-L494 with https://github.com/music-encoding/guidelines/edit/master/_guidelines-dev/04-cmn/01-cmnbasics/04-cmnnoteschords/02-cmnrests.md

[th-we]

Many thanks for the work on this!