Add 'regularize' helper function to all relevant tools, and document (was: HRGT sorting has some unwanted side-effects)

[RieksJ]

The tev2-glossaries-demo has a set of tabs that show different ways in which a glossary can be created. The tab "Latest eSSIF-Lab glossary" (at the right) has the MRGref {% hrg="essif-lab:latest" converter="markdown-section-3" sorter="{{glossaryText}}" %}, which means that the resulting glossary should be ordered according to the glossaryText of its entries. The screenshot below shows that this is not the case:

The reason for this would be that the first two entries start with a termref, i.e., with the [-character. I'm wondering if/what we should do about this.

[RieksJ]

Perhaps we can fix this by having a helper function regularize, that regularizes its argument. Regularizing glossaryText would certainly do the trick. Also, I think that having such a helper function may prove to be useful at some point in other contexts.

[RieksJ]

@Ca5e mentions that having the NoRefs helper function would be what you would really want, and this is true. However, it would require the NoRefs function to know what the TermRef interpreter would be, and then things become difficult.

Decision:

• [x] the regularize function will be added (also in other tools.
• [ ] documentation (tev2-specifications) for the tools (HRGT, TRRT0 will be updated so that they list the helper-functions that they have (so if there are other discrepancies they can be fixed as well)

[Ca5e]

I've added all of the helper functions to the converter profile. @RieksJ please take a look.

[RieksJ]

Good start. @Ca5e, could you:

• [ ] check the example I gave converter profile. Can you pay attention to the int section and see how that is? I was wondering there whether that would be the interpreter as specified (which could be a predefined one) or the actual regex. Does this section allow for one field that has the interpreter 'as specified' (predefined, or regex_ and another one 'as used' (regex only)?
• [ ] the int section needs better explanation. I cannot unambiguously learn whether int.type is the only field in that section, or what that int.type field actually contains.
• [x] The err.file needs better explanation. Is it a path? just a filename? ...? Does this need some rethinking, e.g., taking into account what one might need to construct an error message that can help readers to find out where and what went wrong?
• [ ] Can we add a remedy field to the err section (as an array of strings)? The idea is that if a tool may be able to provide a set of (possible/likely/...) remedies that the err-converter might turn into a usefule message. We could do that even if tools (currently) do not yet provide such remedies; it would to serve tool-creators as a reminder to think harder to make life of the tool-users easier...
• [ ] Pls. review the noRefs helper section as I changed it. Here are some questions I had:
  • the 'type' argument, would that be any of the predefined termref interpreters, or a regex? If so, that should be stated here, and we can simply refer to them them with the termref [xxxx](trrt#predefined-interpreters@).
  • One example you give starts and terminates with /. That is not how regexes are typically passed as an argument within TEv2. The use of that (if it isn't a mistake) should be clarified.
  • I'm not sure what multiple types do. Are they processed in sequence (so all termrefs of the identified types get removed), or have I missed something, or ...?
• [ ] I've also added a section on ifValue; can you verify that this is ok?

[RieksJ]

Since the code was updated, and I myself did documentation updates, I am satisfied to close this issue. If any documentation/code discrepancies exist, they should be addressed in a new issue (that also goes for the request for a 'remedy' feature)