HTML entities do not work in wiki link custom title

[dixslyf]

Describe the bug Using HTML entities in the custom title of a wiki link causes them to be rendered literally.

To Reproduce I am using cedd7d8cde17e0dbe96b2b9d2210d8da7d2e2d01.

Using a non-breaking space ( ) as an example:

[[category-theory-spivak-2014|Spivak (2014)]] analogizes sets to _bins_, stating that "the study of sets is the study of things in bins".

This is rendered as:

Here is the generated HTML:

<p class="mb-3">
      <a href="bibliography/category-theory-spivak-2014.html" class="text-blue-600 mavenLinkBold hover:underline" data-wikilink-type="WikiLinkNormal">Spivak&amp;nbsp;(2014)</a> analogizes sets to <em>bins</em>, stating that “the study of sets is the study of things in bins”.
</p>

Notice that the & is translated to &.

Expected behavior The   is rendered as a non-breaking space.

Using a regular markdown link works as expected and so can be used as a workaround and a reference:

[Spivak (2014)](bibliography/category-theory-spivak-2014.md) analogizes sets to _bins_, stating that "the study of sets is the study of things in bins".

<p class="mb-3">
      <a href="bibliography/category-theory-spivak-2014.html" class="text-blue-600 mavenLinkBold hover:underline">Spivak&nbsp;(2014)</a> analogizes sets to <em>bins</em>, stating that “the study of sets is the study of things in bins”.
</p>

Another workaround is to type in the desired character directly without using an HTML entity.

Desktop (please complete the following information):

• Browser: Firefox 113.0.1

[kukimik]

It looks like an issue with https://github.com/srid/commonmark-wikilink/.

Note that the original commonmark-hs wikilinks extension also exhibits this behavior. I created an issue to see whether this is intentional (and if so, what is the reason behind this decision) or a bug.

Probably changing untokenize to Commonmark.Entity.unEntity in pWikilink should be enough to fix this, once we are sure that the current behavior is wrong. (But I neither tried it out, nor read the code of srid's commonmark-wikilink; I just played for a moment with the commonmark-hs extension.)

[kukimik]

@PlayerNameHere It looks like there is no standard behavior regarding entities in wikilinks (different apps show different behavior, e.g. GitHub vs Obsidian). I thought you may like to join the discussion in https://github.com/jgm/commonmark-hs/issues/105.

Regardless of the result of this discussion, it is not obvious to me whether it is preferable to keep compatibility with commonmark-hs or make our own choice in https://github.com/srid/commonmark-wikilink/. @srid What do you think?

[kukimik]

https://github.com/jgm/commonmark-hs/pull/109 was accepted, so in commonmark-extensions entities are now recognized both in the link title and in the reference. So I think this is the way to go.

I tried the same simple approach (see https://github.com/kukimik/commonmark-wikilink/commit/67798feae682560a8d7d2b8af7be7c122f95fa17), but didn't have time to test it until now. Unfortunately it turned out that this does not work as expected. If an entity of the form &#some_number; is used in the reference part of the wikilink, the part of the text beginning with # is cut out, because it is treated as an anchor. So the fix needs to be more complicated. And thus I'd rather not do it without a test suite (see https://github.com/srid/commonmark-wikilink/issues/1).