search

giellalt / lang-fkv

Finite state and Constraint Grammar based analysers and proofing tools, and language resources for the Kven Finnish language
https://giellalt.uit.no
GNU Lesser General Public License v3.0
2 stars 0 forks source link

After infra reorg this spring, editdistance info comes up as 0 #7

Closed Trondtr closed 4 months ago

Trondtr commented 5 months ago

After make clean in lang-fkv/tools/spellcheckers/, I get the following for make V=1

uit-mac-443 lang-fkv (main)$ make V=1

*** Compiling fkv - Kven Finnish. ***

CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh '/Users/ttr000/git/giellalt/lang-fkv/build-aux/missing' autoconf
Making all in .

*** Compiling fkv - Kven Finnish. ***

CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh '/Users/ttr000/git/giellalt/lang-fkv/build-aux/missing' autoconf
make[1]: Nothing to be done for `all-am'.
Making all in src
Making all in fst
Making all in morphology
make[3]: Nothing to be done for `all'.
Making all in filters
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt area-tags.txt.tmp Area/
if test -f area-tags.txt ; then \
        if ! diff -q area-tags.txt area-tags.txt.tmp ; then \
            mv -f area-tags.txt.tmp area-tags.txt ; \
        else \
            rm -f area-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f area-tags.txt.tmp area-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt derivation-tags.txt.tmp Der/
if test -f derivation-tags.txt ; then \
        if ! diff -q derivation-tags.txt derivation-tags.txt.tmp ; then \
            mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
        else \
            rm -f derivation-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt dialect-tags.txt.tmp Dial/
if test -f dialect-tags.txt ; then \
        if ! diff -q dialect-tags.txt dialect-tags.txt.tmp ; then \
            mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
        else \
            rm -f dialect-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt error-tags.txt.tmp Err/
if test -f error-tags.txt ; then \
        if ! diff -q error-tags.txt error-tags.txt.tmp ; then \
            mv -f error-tags.txt.tmp error-tags.txt ; \
        else \
            rm -f error-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f error-tags.txt.tmp error-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt grammatical-tags.txt.tmp Gram/
if test -f grammatical-tags.txt ; then \
        if ! diff -q grammatical-tags.txt grammatical-tags.txt.tmp ; then \
            mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
        else \
            rm -f grammatical-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt orig_lang-tags.txt.tmp OLang/
if test -f orig_lang-tags.txt ; then \
        if ! diff -q orig_lang-tags.txt orig_lang-tags.txt.tmp ; then \
            mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
        else \
            rm -f orig_lang-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt semantic-tags.txt.tmp Sem/
if test -f semantic-tags.txt ; then \
        if ! diff -q semantic-tags.txt semantic-tags.txt.tmp ; then \
            mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
        else \
            rm -f semantic-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt usage-tags.txt.tmp Use/
if test -f usage-tags.txt ; then \
        if ! diff -q usage-tags.txt usage-tags.txt.tmp ; then \
            mv -f usage-tags.txt.tmp usage-tags.txt ; \
        else \
            rm -f usage-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f usage-tags.txt.tmp usage-tags.txt ; \
    fi
/Library/Developer/CommandLineTools/usr/bin/make  all-am
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt orig_lang-tags.txt.tmp OLang/
if test -f orig_lang-tags.txt ; then \
        if ! diff -q orig_lang-tags.txt orig_lang-tags.txt.tmp ; then \
            mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
        else \
            rm -f orig_lang-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f orig_lang-tags.txt.tmp orig_lang-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt area-tags.txt.tmp Area/
if test -f area-tags.txt ; then \
        if ! diff -q area-tags.txt area-tags.txt.tmp ; then \
            mv -f area-tags.txt.tmp area-tags.txt ; \
        else \
            rm -f area-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f area-tags.txt.tmp area-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt derivation-tags.txt.tmp Der/
if test -f derivation-tags.txt ; then \
        if ! diff -q derivation-tags.txt derivation-tags.txt.tmp ; then \
            mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
        else \
            rm -f derivation-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f derivation-tags.txt.tmp derivation-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt dialect-tags.txt.tmp Dial/
if test -f dialect-tags.txt ; then \
        if ! diff -q dialect-tags.txt dialect-tags.txt.tmp ; then \
            mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
        else \
            rm -f dialect-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f dialect-tags.txt.tmp dialect-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt error-tags.txt.tmp Err/
if test -f error-tags.txt ; then \
        if ! diff -q error-tags.txt error-tags.txt.tmp ; then \
            mv -f error-tags.txt.tmp error-tags.txt ; \
        else \
            rm -f error-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f error-tags.txt.tmp error-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt grammatical-tags.txt.tmp Gram/
if test -f grammatical-tags.txt ; then \
        if ! diff -q grammatical-tags.txt grammatical-tags.txt.tmp ; then \
            mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
        else \
            rm -f grammatical-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f grammatical-tags.txt.tmp grammatical-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt semantic-tags.txt.tmp Sem/
if test -f semantic-tags.txt ; then \
        if ! diff -q semantic-tags.txt semantic-tags.txt.tmp ; then \
            mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
        else \
            rm -f semantic-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f semantic-tags.txt.tmp semantic-tags.txt ; \
    fi
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/extract-tags.sh lexicon-tags.txt usage-tags.txt.tmp Use/
if test -f usage-tags.txt ; then \
        if ! diff -q usage-tags.txt usage-tags.txt.tmp ; then \
            mv -f usage-tags.txt.tmp usage-tags.txt ; \
        else \
            rm -f usage-tags.txt.tmp ; \
        fi ;\
    else \
        mv -f usage-tags.txt.tmp usage-tags.txt ; \
    fi
Making all in phonetics
Making all in .
make[4]: Nothing to be done for `all-am'.
Making all in tests
make[4]: Nothing to be done for `all'.
Making all in syllabification
make[3]: Nothing to be done for `all'.
Making all in orthography
make[3]: Nothing to be done for `all'.
Making all in transcriptions
make[3]: Nothing to be done for `all'.
Making all in tagsets
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in cg3
make[2]: Nothing to be done for `all'.
make[2]: Nothing to be done for `all-am'.
Making all in tools
Making all in tokenisers
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in tests
make[3]: Nothing to be done for `all'.
Making all in analysers
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in shellscripts
make[2]: Nothing to be done for `all'.
Making all in spellcheckers
Making all in filters
cp -f /Users/ttr000/git/giellalt/lang-fkv/./../giella-core/fst-filters/remove-word-boundary.regex remove-word-boundary.regex
"/usr/local/bin/hfst-regexp2fst"  \
       --format=openfst-tropical \
        "--xerox-composition=ON" -v -S -i remove-word-boundary.regex -o remove-word-boundary.hfst
Reading from remove-word-boundary.regex, writing to remove-word-boundary.hfst
Compiling expression #1
Compiling expression #2
cp -f /Users/ttr000/git/giellalt/lang-fkv/src/fst/filters/remove-usage-tags.regex remove-usage-tags.regex
"/usr/local/bin/hfst-regexp2fst"  \
       --format=openfst-tropical \
        "--xerox-composition=ON" -v -S -i remove-usage-tags.regex -o remove-usage-tags.hfst
Reading from remove-usage-tags.regex, writing to remove-usage-tags.hfst
Compiling expression #1
Compiling expression #2
cp -f /Users/ttr000/git/giellalt/lang-fkv/src/fst/filters/remove-error-strings.regex remove-error-strings.regex
"/usr/local/bin/hfst-regexp2fst"  \
       --format=openfst-tropical \
        "--xerox-composition=ON" -v -S -i remove-error-strings.regex -o remove-error-strings.hfst
Reading from remove-error-strings.regex, writing to remove-error-strings.hfst
Compiling expression #1
Compiling expression #2
Making all in weights
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/clean_weight_corpus.sh spellercorpus.raw.txt spellercorpus.clean.txt
Making all in neural
make[3]: Nothing to be done for `all'.
Making all in .
/usr/local/bin/gmkdir -p `dirname .generated/.stamp`
touch .generated/.stamp
/usr/bin/printf "read regex \
            @\"../../src/fst/filters/remove-usage_except_speller-tags.hfst\" \
        .o. @\"../../src/fst/filters/remove-orig_lang-tags.hfst\"        \
        .o. @\"../../src/fst/filters/remove-CLB-strings.hfst\"           \
        .o. @\"../../src/fst/filters/remove-error-strings.hfst\"         \
        .o. @\"../../src/fst/filters/remove-MinusSpell-strings.hfst\"    \
        .o. @\"../../src/fst/filters/remove-PUNCT-strings.hfst\"         \
        .o. @\"../../src/fst/filters/remove-use_marg-strings.hfst\"      \
        .o. @\"../../src/fst/filters/remove-Use_minus_PMatch-tags.hfst\" \
        .o. @\"../../src/fst/filters/remove-Use_minusGC-tags.hfst\"      \
        .o. @\"../../src/fst/filters/remove-Use_GC-strings.hfst\"        \
        .o. @\"../../src/fst/filters/remove-Use_PMatch-strings.hfst\"    \
        .o. @\"../../src/fst/filters/remove-mwe-tags.hfst\"              \
        .o. @\"../../src/fst/filters/remove-use_mt-strings.hfst\"        \
        .o. @\"../../src/fst/analyser-raw-gt-desc.hfst\" \
        .o. @\"../../src/fst/orthography/downcase-derived_proper-strings.compose.hfst\" \
        ;\n\
         save stack .generated/generator-speller-gt-norm.tmp.hfst\n\
         quit\n" | /usr/local/bin/hfst-xfst -p -v --format=foma
Using foma as output handler
Reading from standard input...
warning: both composition arguments contain flag diacritics that are not harmonized
? bytes. 136976 states, 335416 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-speller-gt-norm.tmp.hfst .generated/generator-speller-gt-norm.hfst
cp .generated/generator-speller-gt-norm.hfst generator-speller-gt-norm.hfst
/usr/bin/printf "read regex                                        \
                                                  \
            @\"generator-speller-gt-norm.hfst\"                                                           \
        .o. @\"../../src/fst/filters/remove-hyphenation-marks.hfst\"    \
        .o. @\"../../src/fst/filters/remove-infl_deriv-borders.hfst\"   \
        ; \n\
         twosided flag-diacritics\n\
         save stack .generated/generator-fstspeller-gt-norm.tmp.hfst\n\
         quit\n" | /usr/local/bin/hfst-xfst -p -v --format=foma
Using foma as output handler
Reading from standard input...
? bytes. 135461 states, 335095 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-fstspeller-gt-norm.tmp.hfst .generated/generator-fstspeller-gt-norm.hfst
/usr/local/bin/hfst-fst2fst  -f openfst-tropical .generated/generator-fstspeller-gt-norm.hfst -o .generated/generator-mobilespeller-gt-norm-base.hfst
/usr/local/bin/hfst-reweight -v  \
            -T ./weights/tags.reweight --arcs-only -i .generated/generator-mobilespeller-gt-norm-base.hfst \
        -o .generated/generator-mobilespeller-gt-norm-tag_weighted.hfst
Reading from .generated/generator-mobilespeller-gt-norm-base.hfst, writing to .generated/generator-mobilespeller-gt-norm-tag_weighted.hfst
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 0.000000
only on arc weights, no end states
Reweighting convert(UNNAMED)...
Reading reweights from ./weights/tags.reweight
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 2.000000 for symbol +Cmp
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 1.000000 for symbol +Der/minen
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 50000.000000 for symbol +Use/SpellNoSugg
/usr/bin/printf "\
    set encode-weights ON \n\
    read regex \
        @\"filters/remove-usage-tags.hfst\" \
    .o. @\".generated/generator-mobilespeller-gt-norm-tag_weighted.hfst\" \
    .o. @\"filters/remove-word-boundary.hfst\"  \
    ; \n\
    save stack .generated/generator-mobilespeller-gt-norm.tmp.hfst\n\
    quit\n" | /usr/local/bin/hfst-xfst -p -v
Using default output format OpenFst with tropical weight class
Using OpenFst's tropical weights as output
Reading from standard input...
variable encode-weights = ON
hfst[0]: ? bytes. 131662 states, 334472 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-mobilespeller-gt-norm.tmp.hfst .generated/generator-mobilespeller-gt-norm.hfst
cp .generated/generator-mobilespeller-gt-norm.hfst generator-mobilespeller-gt-norm.hfst
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/make-hfstspeller-version-easter-egg.sh \
        fkv \
        ../.. \
        0.1.1 \
        "mobile" \
        default \
        > easteregg.default.mobile.txt
echo "?*" | /usr/local/bin/hfst-regexp2fst -o .generated/anystar.hfst
cp .generated/anystar.hfst anystar.hfst
sed -e 's/^/nuvviDspeller:/' < easteregg.default.mobile.txt \
        | sed = \
        | sed 'N;s/\n/  /' \
        | perl -pe 's/(.)\t(.+)/\2\t\1/' \
        > easteregg.default.mobile.suggtxt
grep -h -v '^#' words.default.txt easteregg.default.mobile.suggtxt | grep -v '^$'   \
        | /usr/local/bin/hfst-strings2fst  -j \
        -o words.default.mobile.hfst
grep -v '^#' strings.default.txt | grep -v '^$' | cut -f1-2 \
        | /usr/local/bin/hfst-strings2fst  -j \
        | /usr/local/bin/hfst-concatenate   anystar.hfst - \
        | /usr/local/bin/hfst-concatenate - anystar.hfst   \
        -o strings.txt.default.mobile.hfst
cp -f strings.txt.default.mobile.hfst strings.all.default.mobile.hfst
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/editdist.py \
        --verbose \
         \
        --epsilon='@0@' \
        --default-weight=10 \
        --regex \
        --input=editdist.default.txt \
        --output-file=editdist.default.mobile.regex \

/usr/local/bin/hfst-regexp2fst -S  -i editdist.default.mobile.regex\
       --format=openfst-tropical \
        -o .generated/editdist.default.mobile.hfst
cp .generated/editdist.default.mobile.hfst editdist.default.mobile.hfst
/usr/local/bin/hfst-disjunct strings.all.default.mobile.hfst editdist.default.mobile.hfst \
        | /usr/local/bin/hfst-repeat -f 1 -t 2 \
        -o editdist.all.default.mobile.hfst
grep -v '^#' final_strings.default.txt | grep -v '^$' | cut -f1-2 \
        | /usr/local/bin/hfst-strings2fst  -j \
       --format=openfst-tropical \
        -o final_strings.txt.default.mobile.hfst
cp -f final_strings.txt.default.mobile.hfst final_strings.all.default.mobile.hfst
printf "\
        [ @\"words.default.mobile.hfst\"    \
        | \
          [ \
               \
              @\"editdist.all.default.mobile.hfst\"  \
              ( @\"final_strings.all.default.mobile.hfst\" ) \
          ] \
        ];" \
        | /usr/local/bin/hfst-regexp2fst -S -E  \
        | /usr/local/bin/hfst-push-weights --push=initial    \
        | /usr/local/bin/hfst-fst2fst  -f olw   \
          -o errmodel.default.mobile.hfst
/usr/local/bin/hfst-strings2fst  -j < easteregg.default.mobile.txt \
        | /usr/local/bin/hfst-project  --project=lower > easteregg.default.mobile.hfst
/usr/local/bin/hfst-project  \
            -v --project=lower < generator-mobilespeller-gt-norm.hfst \
        | /usr/local/bin/hfst-remove-epsilons  -v                       \
        | /usr/local/bin/hfst-disjunct -v - easteregg.default.mobile.hfst          \
        | /usr/local/bin/hfst-push-weights -v --push=initial           \
        | /usr/local/bin/hfst-fst2fst -v  -f olw          \
        -o acceptor.default.mobile.hfst
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from - and easteregg.default.mobile.hfst, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to acceptor.default.mobile.hfst
Writing Hfst's lookup optimized, weighted format transducers with HFST3 headers
Projecting second <stdin>...
Removing epsilons project-2nd(UNNAMED)...
Disjuncting remove-epsilons(project-2nd(UNNAMED)) and project-2nd(strings(?))...
Pushing towards start union(remove-epsilons(project-2nd(UNNAMED)), project-2nd(strings(?)))...
Converting push-weights-i(union(remove-epsilons(project-2nd(UNNAMED)), project-2nd(strings(?))))...
rm -f fkv-mobile.zhfst
/usr/local/bin/gmkdir -p build/fkv-mobile.zhfst
rm -f build/fkv-mobile.zhfst/*
cp index.mobile.xml build/fkv-mobile.zhfst/index.xml
cp acceptor.default.mobile.hfst build/fkv-mobile.zhfst/acceptor.default.hfst
cp errmodel.default.mobile.hfst build/fkv-mobile.zhfst/errmodel.default.hfst
cd build/fkv-mobile.zhfst/ && /usr/bin/zip -0   ../../fkv-mobile.zhfst *
  adding: acceptor.default.hfst (stored 0%)
  adding: errmodel.default.hfst (stored 0%)
  adding: index.xml (stored 0%)
/usr/local/bin/gmkdir -p 3
cp -f fkv-mobile.zhfst 3/
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/predict.py \
        --verbose \
        --epsilon='@0@' \
        --default-weight=10 \
        --regex \
        --input=editdist.default.txt \
        --output=predict.default.regex
being verbose...
reading from editdist.default.txt...
writing to   predict.default.regex...
"/usr/local/bin/hfst-regexp2fst"  --format=foma \
         -v -S predict.default.regex -o predict.default.hfst
Reading from predict.default.regex, writing to predict.default.hfst
Compiling expression #1
Compiling expression #2
hfst-fst2fst -f olw -v -i predict.default.hfst -o errmodel.default.predict.hfst
Reading from predict.default.hfst, writing to errmodel.default.predict.hfst
Writing Hfst's lookup optimized, weighted format transducers with HFST3 headers
Converting xre(?)...
/usr/local/bin/hfst-fst2fst -f olw -i acceptor.default.mobile.hfst -o acceptor.default.predict.hfst
rm -f fkv-x-predict-mobile.zhfst
/usr/local/bin/gmkdir -p build/fkv-x-predict-mobile.zhfst
rm -f build/fkv-x-predict-mobile.zhfst/*
cp index.predict.xml build/fkv-x-predict-mobile.zhfst/index.xml
cp acceptor.default.predict.hfst build/fkv-x-predict-mobile.zhfst/acceptor.default.hfst
cp errmodel.default.predict.hfst build/fkv-x-predict-mobile.zhfst/errmodel.default.hfst
cd build/fkv-x-predict-mobile.zhfst/ && /usr/bin/zip -0   ../../fkv-x-predict-mobile.zhfst *
  adding: acceptor.default.hfst (stored 0%)
  adding: errmodel.default.hfst (stored 0%)
  adding: index.xml (stored 0%)
/usr/local/bin/gmkdir -p 3
cp -f fkv-x-predict-mobile.zhfst 3/
/usr/local/bin/hfst-invert -v  -i .generated/generator-speller-gt-norm.hfst -o .generated/analyser-speller-gt-norm.hfst
Reading from .generated/generator-speller-gt-norm.hfst, writing to .generated/analyser-speller-gt-norm.hfst
Inverting .generated/generator-speller-gt-norm.hfst...
cp .generated/analyser-speller-gt-norm.hfst analyser-speller-gt-norm.hfst
cp .generated/generator-fstspeller-gt-norm.hfst generator-fstspeller-gt-norm.hfst
/usr/local/bin/hfst-invert -v  -i .generated/generator-fstspeller-gt-norm.hfst -o .generated/analyser-fstspeller-gt-norm.hfst
Reading from .generated/generator-fstspeller-gt-norm.hfst, writing to .generated/analyser-fstspeller-gt-norm.hfst
Inverting .generated/generator-fstspeller-gt-norm.hfst...
cp .generated/analyser-fstspeller-gt-norm.hfst analyser-fstspeller-gt-norm.hfst
/usr/local/bin/hfst-fst2fst --format=openfst-tropical -i .generated/generator-fstspeller-gt-norm.hfst -o .generated/generator-desktopspeller-gt-norm-base.hfst
/usr/local/bin/hfst-reweight -v  \
            -T ./weights/tags.reweight --arcs-only -i .generated/generator-desktopspeller-gt-norm-base.hfst \
        -o .generated/generator-desktopspeller-gt-norm-tag_weighted.hfst
Reading from .generated/generator-desktopspeller-gt-norm-base.hfst, writing to .generated/generator-desktopspeller-gt-norm-tag_weighted.hfst
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 0.000000
only on arc weights, no end states
Reweighting convert(UNNAMED)...
Reading reweights from ./weights/tags.reweight
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 2.000000 for symbol +Cmp
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 1.000000 for symbol +Der/minen
Modifying weights 0.000000 < w < 340282346638528859811704183484516925440.000000 as 1.000000 * id(w) + 50000.000000 for symbol +Use/SpellNoSugg
/usr/local/bin/hfst-strings2fst  -j < easteregg.default.desktop.txt \
        > .generated/easteregg.default.desktop.hfst
/usr/bin/printf "\
    set encode-weights ON \n\
    read regex [ \
        @\"filters/remove-usage-tags.hfst\" \
    .o. @\".generated/generator-desktopspeller-gt-norm-tag_weighted.hfst\" \
    .o. @\"filters/remove-word-boundary.hfst\" ] \
    | @\".generated/easteregg.default.desktop.hfst\" \
    ; \n\
    save stack .generated/generator-desktopspeller-gt-norm.tmp.hfst\n\
    quit\n" | /usr/local/bin/hfst-xfst -p -v
Using default output format OpenFst with tropical weight class
Using OpenFst's tropical weights as output
Reading from standard input...
variable encode-weights = ON
hfst[0]: ? bytes. 131751 states, 334564 arcs, ? paths
hfst[1]: hfst[1]: hfst[1]: .
cp -f .generated/generator-desktopspeller-gt-norm.tmp.hfst .generated/generator-desktopspeller-gt-norm.hfst
cp .generated/generator-desktopspeller-gt-norm.hfst generator-desktopspeller-gt-norm.hfst
/usr/local/bin/hfst-invert -v  -i .generated/generator-desktopspeller-gt-norm.hfst \
    | /usr/local/bin/hfst-prune-alphabet -v \
    | /usr/local/bin/hfst-remove-epsilons -v -o .generated/analyser-desktopspeller-gt-norm.hfst
Reading from .generated/generator-desktopspeller-gt-norm.hfst, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to .generated/analyser-desktopspeller-gt-norm.hfst
Inverting .generated/generator-desktopspeller-gt-norm.hfst...
Pruning invert(UNNAMED)...
Removing epsilons prune-alphabet(invert(UNNAMED))...
cp .generated/analyser-desktopspeller-gt-norm.hfst analyser-desktopspeller-gt-norm.hfst
/Users/ttr000/git/giellalt/lang-fkv/./../giella-core/scripts/editdist.py \
        --verbose \
         \
        --epsilon='@0@' \
        --default-weight=10 \
        --regex \
        --input=editdist.default.txt \
        --output-file=editdist.default.regex \

"/usr/local/bin/hfst-regexp2fst"  --format=foma \
         -v -S editdist.default.regex -o editdist.default.hfst
Reading from editdist.default.regex, writing to editdist.default.hfst
Compiling expression #1
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst

[ then the previous line was repeated 4000 times] <== comment, TT

warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '@_UNKNOWN_SYMBOL_@' is not an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst

[ then the previous line was repeated 40 times ] <== comment, TT

warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
warning: '<CORR> ' is an ordinary symbol in hfst
Compiling expression #2
grep -v '^#' strings.default.txt | grep -v '^$' | cut -f1-2 \
        | /usr/local/bin/hfst-strings2fst  -j \
        | /usr/local/bin/hfst-concatenate   .generated/anystar.hfst - \
        | /usr/local/bin/hfst-concatenate - .generated/anystar.hfst   \
        -o .generated/strings.txt.default.hfst
cp .generated/strings.txt.default.hfst strings.txt.default.hfst
cp -f strings.txt.default.hfst strings.all.default.hfst
/usr/local/bin/hfst-disjunct strings.all.default.hfst editdist.default.hfst \
        | /usr/local/bin/hfst-repeat -f 1 -t 2 \
        -o editdist.all.default.hfst
/usr/local/bin/hfst-disjunct: warning: Transducer type mismatch in strings.all.default.hfst and editdist.default.hfst; using former type as output
grep -v '^#' final_strings.default.txt | grep -v '^$' | cut -f1-2 \
        | /usr/local/bin/hfst-strings2fst  -j \
       --format=openfst-tropical \
        -o .generated/final_strings.txt.default.hfst
cp .generated/final_strings.txt.default.hfst final_strings.txt.default.hfst
cp -f final_strings.txt.default.hfst final_strings.all.default.hfst
printf "\
        [ @\"words.default.hfst\"    \
        | \
          [ \
               \
              @\"editdist.all.default.hfst\"  \
              ( @\"final_strings.all.default.hfst\" ) \
          ] \
        ];" \
        | /usr/local/bin/hfst-regexp2fst -S -E  \
        | /usr/local/bin/hfst-push-weights --push=initial    \
        | /usr/local/bin/hfst-fst2fst  -f olw   \
          -o errmodel.default.hfst
/usr/local/bin/hfst-invert -v  -i .generated/generator-mobilespeller-gt-norm.hfst -o .generated/analyser-mobilespeller-gt-norm.hfst
Reading from .generated/generator-mobilespeller-gt-norm.hfst, writing to .generated/analyser-mobilespeller-gt-norm.hfst
Inverting .generated/generator-mobilespeller-gt-norm.hfst...
cp .generated/analyser-mobilespeller-gt-norm.hfst analyser-mobilespeller-gt-norm.hfst
/usr/local/bin/hfst-compose -1 filters/remove-error-strings.hfst -2 generator-desktopspeller-gt-norm.hfst -F \
        | /usr/local/bin/hfst-project  \
            -v --project=lower \
        | /usr/local/bin/hfst-remove-epsilons  -v                       \
        | /usr/local/bin/hfst-disjunct -v - .generated/easteregg.default.desktop.hfst  \
        | /usr/local/bin/hfst-push-weights -v --push=initial           \
        | /usr/local/bin/hfst-fst2fst -v  -f olw          \
        -o acceptor.default.hfst
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from - and .generated/easteregg.default.desktop.hfst, writing to <stdout>
Reading from <stdin>, writing to <stdout>
Reading from <stdin>, writing to acceptor.default.hfst
Writing Hfst's lookup optimized, weighted format transducers with HFST3 headers
Projecting second compose(UNNAMED, UNNAMED)...
Removing epsilons project-2nd(compose(UNNAMED, UNNAMED))...
Disjuncting remove-epsilons(project-2nd(compose(UNNAMED, UNNAMED))) and strings(?)...
Pushing towards start union(remove-epsilons(project-2nd(compose(UNNAMED, UNNAMED))), strings(?))...
Converting push-weights-i(union(remove-epsilons(project-2nd(compose(UNNAMED, UNNAMED))), strings(?)))...
rm -f fkv.zhfst
/usr/local/bin/gmkdir -p build/fkv.zhfst
rm -f build/fkv.zhfst/*
cp index.xml build/fkv.zhfst/index.xml
cp acceptor.default.hfst build/fkv.zhfst/acceptor.default.hfst
cp errmodel.default.hfst build/fkv.zhfst/errmodel.default.hfst
cd build/fkv.zhfst/ && /usr/bin/zip -9   ../../fkv.zhfst *
  adding: acceptor.default.hfst (deflated 81%)
  adding: errmodel.default.hfst (deflated 80%)
  adding: index.xml (deflated 55%)
/usr/local/bin/gmkdir -p 3
cp -f fkv.zhfst 3/
ln -f fkv.zhfst fkv-desktop.zhfst
rm editdist.all.default.mobile.hfst .generated/analyser-speller-gt-norm.hfst final_strings.txt.default.hfst strings.txt.default.hfst editdist.default.mobile.hfst strings.txt.default.mobile.hfst predict.default.regex .generated/strings.txt.default.hfst final_strings.all.default.mobile.hfst .generated/editdist.default.mobile.hfst editdist.default.mobile.regex .generated/analyser-fstspeller-gt-norm.hfst words.default.mobile.hfst final_strings.all.default.hfst strings.all.default.mobile.hfst predict.default.hfst easteregg.default.mobile.suggtxt .generated/final_strings.txt.default.hfst strings.all.default.hfst final_strings.txt.default.mobile.hfst .generated/generator-speller-gt-norm.hfst
Making all in hyphenators
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in grammarcheckers
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in tests
make[3]: Nothing to be done for `all'.
Making all in mt
Making all in filters
make[3]: Nothing to be done for `all'.
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in apertium
Making all in filters
make[4]: Nothing to be done for `all'.
Making all in tagsets
make[4]: Nothing to be done for `all'.
Making all in .
make[4]: Nothing to be done for `all-am'.
Making all in cgbased
Making all in .
make[4]: Nothing to be done for `all-am'.
Making all in tts
Making all in .
make[3]: Nothing to be done for `all-am'.
Making all in test
make[3]: Nothing to be done for `all'.
make[2]: Nothing to be done for `all-am'.
Making all in docs
make[1]: Nothing to be done for `all'.
Making all in test
Making all in src
Making all in phonology
make[3]: Nothing to be done for `all'.
Making all in orthography
make[3]: Nothing to be done for `all'.
Making all in morphology
make[3]: Nothing to be done for `all'.
Making all in syntax
make[3]: Nothing to be done for `all'.
make[3]: Nothing to be done for `all-am'.
Making all in tools
Making all in spellcheckers
Making all in fstbased
Making all in desktop
Making all in hfst
make[6]: Nothing to be done for `all'.
make[6]: Nothing to be done for `all-am'.
Making all in mobile
make[6]: Nothing to be done for `all-am'.
make[5]: Nothing to be done for `all-am'.
make[4]: Nothing to be done for `all-am'.
Making all in mt
Making all in apertium
make[4]: Nothing to be done for `all'.
make[4]: Nothing to be done for `all-am'.
Making all in hyphenators
Making all in fstbased
make[4]: Nothing to be done for `all'.
Making all in patternbased
make[4]: Nothing to be done for `all'.
make[4]: Nothing to be done for `all-am'.
make[3]: Nothing to be done for `all-am'.
make[2]: Nothing to be done for `all-am'.
flammie commented 5 months ago

Compiling expression #1 warning: ' ' is an ordinary symbol in hfst warning: ' ' is an ordinary symbol in hfst

It works on my linux and compilation is mostly the same but this doesn't appear...

flammie commented 5 months ago

newest giella-core should compile more with openfst and in hidden work dir