search

singnet / language-learning

OpenCog Unsupervised Language Learning
https://wiki.opencog.org/w/Language_learning
MIT License
32 stars 11 forks source link

Parse-evaluator "-i" mode works opposite as intended #248

Closed glicerico closed 5 years ago

glicerico commented 5 years ago

According to reports from Sergey's team, the "alternative" parse evaluator option "-i" works opposite than is expected; this means, when using "-i" the LEFT-WALL and final dot are considered in the evaluation (instead of ignored), and if the "-i" is not used, they are ignored. I could not reproduce this behavior in my system. Example:

Thanks for help! I removed the flag '-i' and got the desired result. It `s Magic!

(ull) leto@leto-desktop:~/projects/structural-probes-as-a-service$ parse-evaluator -a -r pipeline/gold/gold_noQuotes_manual.ull -t pipeline/parses_all_70/sumabs_v4_gold.ipw.ull Code writen for Python3.6.4. Using: 3.6.6 Finished loading Finished loading

Avg Precision: 67.28% Avg Recall: 65.27% Avg Fscore: 66.26%

A total of 229 parses evaluated, 100.00% of reference file 3.00 ignored links per evaluated parse

alexei-gl commented 5 years ago

@glicerico , I can not reproduce the error. "Ignored links" is zero without "-i" and greater than zero with "-i" set. Please, make sure you have the latest source code.