Add default compiler options to all languages

giellalt / template-lang-und

A template repo for new languages, as well as to update existing language repos with.

https://giellalt.uit.no/

GNU Lesser General Public License v3.0

2 stars 1 forks source link

Add default compiler options to all languages #2

Closed snomos closed 4 years ago

snomos commented 4 years ago

It would be useful to have a section in configure.ac to set language specific compiler and optimisation options. Relevant options are:

foma=yes/no (by default, that is)
hfst=yes/no
xfst=yes/no
hyperminimisation=yes/no
reversed-intersect=yes/no
possibly also hfst-backend-format, although we'll probably just make it foma everywhere, for fastest compilation

The idea is to define a section with suitable variables in configure.ac and then reference these variables in the relevant places in the m4/*.m4 files. This way we can tune the default compilation options for each language, while at the same time keep the general configuration files language independent.

This was first discussed in giellalt/lang-crk#3.

snomos commented 4 years ago

The initial default values should reflect the defaults we want for most languages, which are:

foma=no
hfst=yes
xfst=no
hyperminimisation=no
reversed-intersect=no? but maybe yes, as that works best for suffix heavy languages where the morphophonology is to a large degree happening in the interplay between stems and suffixes. In those cases, reversing the fst means that unique contexts can be found early on, which dramatically speeds up the intersection process. The majority of languages being worked actively on are of this type, so maybe yes is best as the initial default value.

Another compilation option is alignment, but that one is already set to yes for all languages, and should probably just stay like that. If needed, we can add an option for that one later on.

And as mentioned above, we should just make the hfst-backend=foma as a default everywhere.

flammie commented 4 years ago

I have added a place for some default options finally. sorry for the delay I had to tread carefully for the first template merge.

For the larger theme of making the infra more like any modern software engineering project / framework, it is my long term goal as well.

aarppe commented 4 years ago

As for XFST, in the case of crk, the XFST implementation of TWOLC, the twolc compiler, did not work properly with the morphophonology, so we have had to hfst-twolc. In the case of srs, the XFST compiler xfst breaks with twosided flag-diacritics, so there as well we have had to use hfst-xfst. Maybe XFST works as it should for other languages, and it certainly is faster than HFST, but since XFST is known to not to work, I'm not convinced we really should be promoting its use at all anymore.

snomos commented 3 years ago

As for XFST, in the case of crk, the XFST implementation of TWOLC, the twolc compiler, did not work properly with the morphophonology, so we have had to hfst-twolc. In the case of srs, the XFST compiler xfst breaks with twosided flag-diacritics, so there as well we have had to use hfst-xfst. Maybe XFST works as it should for other languages, and it certainly is faster than HFST, but since XFST is known to not to work, I'm not convinced we really should be promoting its use at all anymore.

I suggest we move such general discussions to Zulip. I will reply there.

snomos commented 3 years ago

Discussion continued at https://giella.zulipchat.com/#narrow/stream/124585-langs-infra/topic/Default.20compiler.20options