ygojson / ygojson-tools

Tooling for the ygojson project
MIT License
1 stars 0 forks source link

Use a wikitext parser to improve the pattern-base mapping #119

Closed magicDGS closed 5 months ago

magicDGS commented 6 months ago

After some weeks of research on wikitext parsers, looks like the java version for most of them is deprecated or out of maintenance. Also, doing our own parser with some generator providing our own lexer/grammar (i.e., ANTLR) is a no-go as the wikitext syntax has the same problem as Markdown and cannot be properly described.

Nevertheless, most of the Wikipedia/Wikitext tooling out there uses Sweble anyway with some workarounds. For example, dkpro-jwpl creates a shaded artifact to substitute the javax for the jakarta namespace to use a different version.

After some analysis of Sweble for our use-case, looks like we shouldn't use anyway the full engine but just the low-level parser (a PR with the PoC and/or branch is coming soon). This means that we might don't have the problem with jakarta and/or we just need to be sure that is not polluting our dependencies.

Thus, the idea is the following:

Currently, the following use-cases must be migrated to using a parser to simplify the code (and make sure that the expectations from unit/approval tests are still met):

magicDGS commented 5 months ago

Discarded as this was already refactor in #122 as part of #120. We would try to go with the regex as it is simpler to implement, debug and the sweble parser is outdated and not maintained.