divvun / libdivvun

lib for running gramcheck and other pipelines + cli; modules for CG→spelling, CG→feedback, tagging blanks
https://giellalt.github.io/proof/gramcheck/GrammarCheckerDocumentation.html
GNU General Public License v3.0
9 stars 1 forks source link

Error messages do not work for Finnish #4

Closed snomos closed 6 years ago

snomos commented 6 years ago

To repeat, do:

echo "Hän  ei tuleee vielä..." | hfst-tokenise -g tokeniser-gramcheck-gt-desc.pmhfst | divvun-blanktag analyser-gt-whitespace.hfst | divvun-cgspell fi.zhfst | vislcg3 -g grammarchecker.bin | divvun-suggest -g generator-gt-norm.hfstol -m errors.xml -j

Result:

WARNING: No message for "double-space-before"
WARNING: No message for "typo"

despite both having messages defined in errors.xml.

unhammer commented 6 years ago

What does

echo "Hän  ei tuleee vielä..." | hfst-tokenise -g tokeniser-gramcheck-gt-desc.pmhfst | divvun-blanktag analyser-gt-whitespace.hfst | divvun-cgspell fi.zhfst | vislcg3 -g grammarchecker.bin

give?

Can you upload errors.xml and generator-gt-norm.hfstol here? (maybe zip first, or use filebin.net)

snomos commented 6 years ago

Limited to 10 suggestions, I get:

$ echo "Hän  ei tuleee vielä..." | hfst-tokenise -g tokeniser-gramcheck-gt-desc.pmhfst | divvun-blanktag analyser-gt-whitespace.hfst | divvun-cgspell -n 10 fi.zhfst | vislcg3 -g grammarchecker.bin
"<Hän>"
    "hän" Pron Pers Sg3 Nom <W:0.0000000000>
:  
"<ei>"
    "ei" V Neg Act Sg3 <W:0.0000000000> <doubleSpaceBefore> &double-space-before
: 
"<tuleee>"
    "tuleee" ?
    "tulla" V Act Ind Prs Sg3 <W:10> <WA:0> <spelled> "<tulee>" &SUGGESTWF &typo &err_neg_posv
    "eve" N Sg Nom <W:10> <WA:0> <spelled> "<tuleeve>" &SUGGESTWF &typo
        "tule" N Sg Nom Use/NoHyphens
    "eve" N Sg Nom <W:10> <WA:0> <spelled> "<tuleeve>" &SUGGESTWF
        "tule" N Sg Nom
    "tee" N Sg Nom <W:10> <WA:0> <spelled> "<tuletee>" &SUGGESTWF &typo
        "tule" N Sg Nom Use/Hyphen
    "tee" N Sg Nom <W:10> <WA:0> <spelled> "<tuletee>" &SUGGESTWF
        "tule" N Sg Nom Use/NoHyphens
    "tee" N Sg Nom <W:10> <WA:0> <spelled> "<tuletee>" &SUGGESTWF
        "tule" N Sg Nom
    "ewe" N Sg Nom <W:10> <WA:0> <spelled> "<tuleewe>" &SUGGESTWF &typo
        "tule" N Sg Nom Use/NoHyphens
    "ewe" N Sg Nom <W:10> <WA:0> <spelled> "<tuleewe>" &SUGGESTWF
        "tule" N Sg Nom
    "tulla" V Act Pot Sg3 <W:10> <WA:0> <spelled> "<tullee>" &SUGGESTWF &typo &err_neg_posv
    "ele" N Sg Nom <W:10> <WA:0> <spelled> "<tuleele>" &SUGGESTWF &typo
        "tule" N Sg Nom Use/NoHyphens
    "ele" N Sg Nom <W:10> <WA:0> <spelled> "<tuleele>" &SUGGESTWF
        "tule" N Sg Nom
    "tule" N Sg Ill <W:10> <WA:0> <spelled> "<tuleen>" &SUGGESTWF &typo
    "tuli" N Sg Ill <W:10> <WA:0> <spelled> "<tuleen>" &SUGGESTWF &typo
    "pee" N Sg Nom <W:10> <WA:0> <spelled> "<tulepee>" &SUGGESTWF &typo
        "tule" N Sg Nom Use/Hyphen
    "pee" N Sg Nom <W:10> <WA:0> <spelled> "<tulepee>" &SUGGESTWF
        "tule" N Sg Nom Use/NoHyphens
    "pee" N Sg Nom <W:10> <WA:0> <spelled> "<tulepee>" &SUGGESTWF
        "tule" N Sg Nom
    "ee" Interj <W:20> <WA:0> <spelled> "<aueeee>" &SUGGESTWF &typo
        "ee" Interj
            "au" Interj
    "ee" Pcle <W:20> <WA:0> <spelled> "<aueeee>" &SUGGESTWF &typo
        "ee" Interj
            "au" Interj
    "ee" Pcle <W:20> <WA:0> <spelled> "<aueeee>" &SUGGESTWF
        "ee" Interj
            "au" Pcle
    "ee" Interj <W:20> <WA:0> <spelled> "<aueeee>" &SUGGESTWF
        "ee" Interj
            "au" Pcle
    "ee" Interj <W:20> <WA:0> <spelled> "<tseee>" &SUGGESTWF &typo
        "tse" Interj
    "ee" Pcle <W:20> <WA:0> <spelled> "<tseee>" &SUGGESTWF &typo
        "tse" Interj
    "ee" Pcle <W:20> <WA:0> <spelled> "<tseee>" &SUGGESTWF
        "tse" Pcle
    "ee" Interj <W:20> <WA:0> <spelled> "<tseee>" &SUGGESTWF
        "tse" Pcle
: 
"<vielä>"
    "vielä" Pcle <W:0.0000000000>
"<.>"
    "." Punct <W:0.0000000000>

"<..>"
    ".." Err/Orth Punct <W:0.0000000000>
    ".." Punct <W:0.0000000000>
:\n
snomos commented 6 years ago

generator and errors.xml attached. errors.xml.zip generator-gt-norm.hfstol.gz

unhammer commented 6 years ago
unhammer commented 6 years ago

9d0ca478c1b3810a39e15872dcf7d8066aa4f729 should fix this