Error when template tag contains {{!}}

[chrismytton]

Problem

When a template tag that's being parsed contains a sub-template, e.g. {{!}}, it gets picked up by the regular expression we use to match template tags, which means we end up with a partial match.

Example

{{Example|foo={{!}}|bar=baz}}

Currently matching the above template with the regular expression returns these parts:

{"before"=>"", "template_name"=>"Example", "parameters"=>"|foo={{!", "after"=>"|bar=baz}}"}

Whereas it should actually return:

{"before"=>"", "template_name"=>"Example", "parameters"=>"|foo={{!}}|bar=baz", "after"=>""}

Possible solution

• Copy Listeria's approach, which is to explicitly match one of the arguments after the sparql parameter, thus sidestepping any }} within the SPARQL query. This works, but is a bit hacky because it depends on the parameters being in a certain order.
• Use a negative lookbehind to check for a ! before the }}
• Switch to using a proper parser, rather than regular expressions, as suggested in #2

[chrismytton]

I've opened a pull request with a failing test demonstrating this problem here: https://github.com/everypolitician/mediawiki-page-replaceable_content/pull/6.

[chrismytton]

Use a negative lookbehind to check for a ! before the }}

I've updated https://github.com/everypolitician/mediawiki-page-replaceable_content/pull/6 to do a negative lookbehind, which will at least mean we can make some progress on https://github.com/everypolitician/compare_with_wikidata/issues/40, even if it doesn't fully close this ticket.

[lucychambers]

@chrismytton - is this ticket relevant any more or can it be closed?

[chrismytton]

@lucychambers This ticket still represents a bug in this library, so I think it should stay open.

---

For future reference we fixed the downstream issue that uncovered this problem, https://github.com/everypolitician/compare_with_wikidata/issues/40, by taking a completely different approach to parsing prompt options.

Instead of using this library and trying to parse the options out of a template tag we are instead using subpages at known locations. So we're using a /sparql subpage and fetching the unescaped wikitext, which then takes care of expanding the {{!}} templates as well as any other templates the user wants to use. You can see the initial implementation of this in https://github.com/everypolitician/compare_with_wikidata/pull/42.