Revamp of config creation and tokenization

projectEndings / staticSearch

A codebase to support a pure JSON search engine requiring no backend for any XHTML5 document collection

https://endings.uvic.ca/staticSearch/docs/index.html

Mozilla Public License 2.0

46 stars 21 forks source link

Revamp of config creation and tokenization #284

Closed joeytakeda closed 4 months ago

joeytakeda commented 7 months ago

Major revamp of create_config.xsl and tokenize.xsl to use the tunnelled parameter approach (as detailed in #219 — see also the documentation in tokenize.xsl) This should resolve the two tickets to do with major bugs in the config generation (#219 and #246).

Note that this PR also does the initial removal of Dictionary file support (per #278), but does not change anything in the schema nor in the documentation (so #278 should remain open until those parts are completed, too)

martindholmes commented 4 months ago

Tested this with the EB11 project and all seems good.