search

giellalt / lang-kal

Finite state and Constraint Grammar based analysers and proofing tools, and language resources for the Kalaallisut (Greenlandic) language
https://giellalt.uit.no
GNU General Public License v3.0
10 stars 3 forks source link

Mobile speller killed in CI/CD #1

Closed snomos closed 6 months ago

snomos commented 7 months ago

KAL has not built properly in a while. The log file says this:

/usr/bin/hfst-compose  -F \
.generated/generator-mobilespeller-gt-norm-base.hfst mob_spellercorpus.surfweights.hfst \
-o .generated/generator-mobilespeller-gt-norm-freq_weighted.hfst
make[3]: *** [Makefile:2496: .generated/generator-mobilespeller-gt-norm-freq_weighted.hfst] Killed

I suspect it is an out-of-memory thing, KAL is known to be very memory hungry during compilation. Give the builder VM more RAM?

dylanhand commented 6 months ago

I increased linux-02's memory to 28,000MiB from 24,000MiB and the build now succeeds.

snomos commented 6 months ago

Thanks, I have noticed the same. I consider this bug fixed.

TinoDidriksen commented 6 months ago

If not done already, I would also install libgoogle-perftools-dev and export LD_PRELOAD=libtcmalloc_minimal.so to drastically speed up the builds. See https://github.com/hfst/hfst/issues/507 for stats.

dylanhand commented 6 months ago

If not done already, I would also install libgoogle-perftools-dev and export LD_PRELOAD=libtcmalloc_minimal.so to drastically speed up the builds. See hfst/hfst#507 for stats.

@snomos if this is something you'd like, please add it to https://github.com/orgs/divvun/projects/2

snomos commented 6 months ago

@snomos if this is something you'd like, please add it to https://github.com/orgs/divvun/projects/2

Done.