Finnish hyphenation

[bertfrees]

There are several options for implementing a hyphenator. The two technologies supported in the Pipeline are TeX and libhyphen (but we don't have to limit ourselfs to these two).

• TeX patterns for Finnish are already available in Dotify here and here. The patgen tool can be used to generate additional patterns (possibly in an automatic fashion based on a corpus of correctly hyphenated words)
• I haven't been able to find a libhyphen dictionary in the LibreOffice repository, but libhyphen patterns can be automatically derived from TeX patterns. A benefit of using libhyphen instead of TeX is that you can do non-standard hyphenation, and that the patterns can possibly be used by liblouis (the nocross rule)

Once we have decided upon the technology there is still the question of where to put the resources (the files with hyphenation patterns):

• in TeX (upstream)
• in LibreOffice (upstream)
• in liblouis
• in pipeline-mod-braille
• in pipeline-mod-celia
• in Dotify

Tests can be put either in one of these projects, or in functional-testing.

Finally, running tests can be done in multiple ways:

• using the libhyphen API (C)
• using the jhyphen API (Java)
• using the libhyphen module of pipeline-mod-braille
• using the liblouis-java API
• using the liblouis module of pipeline-mod-braille
• using the texhyphj API
• using the texhyph module of pipeline-mod-braille
• using the Dotify API
• using the dotify module of pipeline-mod-braille

[bertfrees]

We ended up generating both a libhyphen and a tex patterns file from the same source. The source code lives in pipeline-mod-celia. Tests are run with both the texhyphj API, using the texhyph module of pipeline-mod-braille, and using the libhyphen module of pipeline-mod-braille. The texhyph module of pipeline-mod-braille needs to be fixed, see issue https://github.com/snaekobbi/pipeline-mod-braille/issues/44.

[bertfrees]

The texhyph module of pipeline-mod-braille is now fixed, see snaekobbi/pipeline-mod-braille#44 and https://github.com/snaekobbi/pipeline-mod-celia/commit/1c18b6189c4dce7cbe182022efdf53e9f8616bfa.