search

giellalt / bugzilla-dummy

0 stars 0 forks source link

After hfst update 2014-03-27 several languages do not compile on Mac; segmentation fault 11 (Bugzilla Bug 1844) #1526

Closed albbas closed 10 years ago

albbas commented 10 years ago

This issue was created automatically with bugzilla2github

Bugzilla Bug 1844

Date: 2014-03-28T06:46:20+01:00 From: Jack Rueter <> To: Tomi Pieski <> CC: sjur.n.moshagen, tommi.pirinen, trond.trosterud

Last updated: 2014-04-22T10:40:05+02:00

albbas commented 10 years ago

Comment 9221

Date: 2014-03-28 06:46:20 +0100 From: Jack Rueter <>

hfst3 At revision 3844 Mac OS 10.8.5

cd main/langs/vro make clean

./autogen.sh && ./configure --with-hfst --enable-spellers make

++ MMASECTION...5, ENDLEX...1 Building lexicon...Minimizing...Done! SOURCE: 112.5 Kb. 1593 states, 2432 arcs, Circular.

lexc> save-source transcriptor-numbers2text-raw.xfst opening "transcriptor-numbers2text-raw.xfst" Opening 'transcriptor-numbers2text-raw.xfst'... Done.

lexc> quit Making all in tagsets make[2]: Nothing to be done for `all'. Making all in . INTRSCT generator-raw-gt-desc.tmp.hfst GEN generator-raw-gt-desc.hfst RGX2FST analyser-gt-desc.tmp.hfst GEN analyser-gt-desc.hfst /bin/sh: line 1: 1686 Done /usr/local/bin/hfst-invert analyser-gt-desc.tmp.hfst 1687 Segmentation fault: 11 | /usr/local/bin/hfst-fst2fst -f olw -o analyser-gt-desc.hfst make[2]: [analyser-gt-desc.hfst] Error 139 make[1]: [all-recursive] Error 1 make: *** [all-recursive] Error 1

++

albbas commented 10 years ago

Comment 9222

Date: 2014-03-28 07:25:14 +0100 From: Sjur Nørstebø Moshagen <>

$ hfst-info No tests selected; printing known data HFST info version: 0.1 HFST packaging: hfst 3.6.1 HFST version: 3.6.1 HFST long version: 300060001 HFST configuration revision: $Revision: 3721 $

All is ok.

$ hfst-info No tests selected; printing known data HFST info version: 0.1 HFST packaging: hfst 3.6.1 HFST version: 3.6.1 HFST long version: 300060001 HFST configuration revision: $Revision: 3844 $

That is, I am not able to reproduce the bug, neither with hfst 3721 nor with hfst 3844. This is all done on MacOSX 10.9.2, using the following hfst configuration:

./configure --with-unicode-handler=ICU --enable-all-tools --enable-silent-rules CXXFLAGS="-I/usr/include/c++/4.2.1" LDFLAGS=" -stdlib=libstdc++"

Tested with vro.

What other languages did you test?

albbas commented 10 years ago

Comment 9226

Date: 2014-03-28 09:06:30 +0100 From: Jack Rueter <>

(In reply to comment #1)

$ hfst-info No tests selected; printing known data HFST info version: 0.1 HFST packaging: hfst 3.6.1 HFST version: 3.6.1 HFST long version: 300060001 HFST configuration revision: $Revision: 3721 $

All is ok.

$ hfst-info No tests selected; printing known data HFST info version: 0.1 HFST packaging: hfst 3.6.1 HFST version: 3.6.1 HFST long version: 300060001 HFST configuration revision: $Revision: 3844 $

That is, I am not able to reproduce the bug, neither with hfst 3721 nor with hfst 3844. This is all done on MacOSX 10.9.2, using the following hfst configuration:

./configure --with-unicode-handler=ICU --enable-all-tools --enable-silent-rules CXXFLAGS="-I/usr/include/c++/4.2.1" LDFLAGS=" -stdlib=libstdc++"

Tested with vro.

What other languages did you test?

Mac OS 10.9.2

cd main svn up cd langs/vro make clean ./autogen.sh ./configure --with-hfst make

make check +++ YAML test 1: ./A-peraedue3_gt-norm.yaml + analyser-gt-norm.hfst - Error: LookupError("['/usr/local/bin/hfst-lookup'] had an error:\n/usr/local/bin/hfst-lookup: ./../../../src/analyser-gt-norm.hfst is not a valid transducer file\n(Error code: 1)",) To rerun with more details, please triple-click, copy and paste the following:

pushd /Users/jackrueter/Languages/gtsvn/langs/vro/test/src/morphology; /opt/local/bin/python3.2 /Users/jackrueter/Languages/gtsvn/gtcore/scripts/morph-test.py -c -i -S hfst --app /usr/local/bin/hfst-lookup --morph ./../../../src/analyser-gt-norm.hfst --gen ./../../../src/generator-gt-norm.hfst ./A-peraedue3_gt-norm.yaml; popd

YAML test 1: ./A-peraedue3_gt-norm.yaml + analyser-gt-norm.xfst - PASS +++ SINCE THIS FAILS Recheck hfst3

cd hfst3 svn up revision 3844 ./autogen.sh ./configure --with-unicode-handler=ICU --enable-all-tools --enable-silent-rules CXXFLAGS="-I/usr/include/c++/4.2.1" LDFLAGS=" -stdlib=libstdc++" make ./scripts/generate-cc-files.sh make +++

include <tr1/unordered_map>

     ^

171 warnings and 1 error generated. make[6]: [hfst-tag.o] Error 1 make[5]: [all-recursive] Error 1 make[4]: [all-recursive] Error 1 make[3]: [all] Error 2 make[2]: [all-recursive] Error 1 make[1]: [all-recursive] Error 1 make: *** [all] Error 2

+++

Now nothing generates, but it must be separate issues on both of my machines This Mavericks is an upgrade. I will add to this from my other mac 10.8.5 a little later. +++

albbas commented 10 years ago

Comment 9227

Date: 2014-03-28 09:45:03 +0100 From: Sjur Nørstebø Moshagen <>

Testing yrk, I am able to reproduce the bug. But yrk has several issues:

1)

XFST yrk.lexc.xfst

Warning: Ignoring info strings. Warning: Sublexicons defined but not used: POSTP_ NUMBERSUF PRON DEM PERS Abbreviation Acronym A_ТЁН A_НЮБЕ-Pal/Var A_Cons AVow ADV DESCR_ N_СЁЁ N_ХАБИЕ N_НЮБЕ-Pal/Var/ӇУВО TEST_NGAE N_МАНСО N_ӇАВО N_ПЕНЗЕРЕ NMN_Ё NMN_ПИ NMN_ТЫ NMN_ХАВО NMN_СЁЁ N_ЯЙ N_ИБЯ N_ConsO N_ТЭНО N-tojo N-xawo N-якэ N-N SGPROSSUF_АМНА SG-LOC POSSESSA-SG-PROS_увна POSSESSA-PL-ACCGEN_PXSG1 NUM_ӇУДИ NUM_ЕД IV_Е IV_МЫ TV_МЫ IV_НИ IV_НО VR_НУ TV_НУ TV_ТУ IV_НЕНЫ TV_ӇЭСУ TV_ЮХУ TV_ЛАХАНА-Pal/Var TV_ЯˮАВЛУ IV_НЭКАЛ V-01_ТО V-01_МИС IND-4-PRT1-DU3_DIAL/W V_Mod/int-OPL V_Mod/int-NON-OPL SG3-PRED-AOR/PRET1 Warning: Lost all entries containing the following symbols: ' / K r ³ ² +Ord +ABBR +ACR ӥ Б Г Д Е Ж З Й К Н О Т У Х Ц Ч Ш Щ Ъ Ы Ь Э Ё І Ӧ щ і ӧ

Lost entries containing regular alphabetic symbols look like a serious issue. Note that these warnings come from Xerox.

2)

LEXC transcriptor-clock2text-raw.xfst lexc> compile-source clock.lexc opening "clock.lexc" Opening 'clock.lexc'... Root...3, BEFORE...3, beforesmallhours...9, beforelargehours...15, REST...48, BEFpunkt...2, AFTpunkt...3, HOUR-30...33, WHOLE-HOUR...1, HALF-HOUR...1, H-MAFT_minutes...59, ANDMINUTES...1, BEF...29, TO...1, HOUR...12, HOUR-FINAL...1 Building lexicon... ERROR: Sublexicon is mentioned but not defined. (H-MAFT)

Again, this is Xerox. It should have stopped the build, but didn't (only xfst does that, not lexc).

3)

Your reported issue:

RGX2FST analyser-gt-norm.tmp.hfst GEN analyser-gt-norm.hfst /bin/sh: line 1: 66346 Done /usr/local/bin/hfst-invert analyser-gt-norm.tmp.hfst 66347 Segmentation fault: 11 | /usr/local/bin/hfst-fst2fst -f olw -o analyser-gt-norm.hfst make[2]: *** [analyser-gt-norm.hfst] Error 139

I will rerun with 'make V=1' to get more details.

albbas commented 10 years ago

Comment 9236

Date: 2014-03-29 11:20:59 +0100 From: Jack Rueter <>

Perhaps this is actually a Java issue:

I searched java and the first suggestion GOOGLE gave me for continuation was: java update for Maverick

http://www.cs.uml.edu/~msherman/AI1_Mavericks.html

I have now updated my home mac OS 10.9.2 and things appear to build.

Now I have to try out the second mac OS 10.9.2 and UBUNTU 2 to see whether the problem is the same.

albbas commented 10 years ago

Comment 9304

Date: 2014-04-18 21:27:54 +0200 From: Sjur Nørstebø Moshagen <>

This is an hfst issue, and relatively new. At least hfst3 svn rev. 3829 is fine. Will test further.

albbas commented 10 years ago

Comment 9305

Date: 2014-04-18 23:11:20 +0200 From: Sjur Nørstebø Moshagen <>

Hfst Revision: 3839 is fine.

albbas commented 10 years ago

Comment 9313

Date: 2014-04-20 21:46:23 +0200 From: Sjur Nørstebø Moshagen <>

Revision: 3840 is NOT fine.

albbas commented 10 years ago

Comment 9314

Date: 2014-04-21 10:03:01 +0200 From: Sjur Nørstebø Moshagen <>

Upon further inspection, Revision: 3839 is NOT fine either.

Procedure to test:

In hfst3:

In $GTHOME/langs/vro/

Result:

Making all in . INTRSCT generator-raw-gt-desc.tmp1.hfst RGX2FST generator-raw-gt-desc.tmp.hfst GEN generator-raw-gt-desc.hfst RGX2FST analyser-gt-desc.tmp.hfst GEN analyser-gt-desc.hfst /bin/sh: line 1: 93965 Done /usr/local/bin/hfst-invert analyser-gt-desc.tmp.hfst 93966 Segmentation fault: 11 | /usr/local/bin/hfst-fst2fst -f olw -o analyser-gt-desc.hfst make[2]: *** [analyser-gt-desc.hfst] Error 139

albbas commented 10 years ago

Comment 9315

Date: 2014-04-21 10:19:21 +0200 From: Jack Rueter <>

Mavericks hfst3 -r3839

./configure --enable-lexc --enable-proc --enable-xfst --without-sfst CXXFLAGS="-I/usr/include/c++/4.2.1"

olo and yrk do not seem to compile. after reverting to -r3829. they compile very (relatively) quickly.

albbas commented 10 years ago

Comment 9316

Date: 2014-04-21 10:47:40 +0200 From: Sjur Nørstebø Moshagen <>

Revision: 3832

is fine. Which means that the bug was introduced in revision 3835:

r3835 | hardwick | 2014-03-20 09:16:26 +0200 (tor, 20 mar 2014) | 2 lines Changed paths: M /trunk/hfst3/libhfst/src/implementations/optimized-lookup/transducer.cc M /trunk/hfst3/libhfst/src/implementations/optimized-lookup/transducer.h

Reinstate special ascii tokenization while avoiding shadowing longer symbols

albbas commented 10 years ago

Comment 9317

Date: 2014-04-21 13:21:45 +0200 From: Sjur Nørstebø Moshagen <>

The bug is reported to the hfst team at:

https://sourceforge.net/p/hfst/bugs/235/

albbas commented 10 years ago

Comment 9340

Date: 2014-04-22 10:40:05 +0200 From: Sjur Nørstebø Moshagen <>

The bug is closed as invalid - it is not in the GTDivvun infra. The hfst team has been notified, and there are at least two bug reports on it.