search

moses-smt / mosesdecoder

Moses, the machine translation system
http://www.statmt.org/moses
GNU Lesser General Public License v2.1
1.58k stars 778 forks source link

kbmira with meteor-1.5.jar is broken #132

Closed raingo closed 9 years ago

raingo commented 9 years ago

OS: linux 64 bit gcc version: 4.4.7 Moses version: latest github 384e0b06d298fe308bd39c6c567f9cf93f42e074

The following command, as part of the mert tunning,

/path/to/kbmira -J 200 --sctype METEOR --scconfig jar:path-to-meteor.jar --dense-init run1.dense  --ffile run1.features.dat --scfile run1.scores.dat -o mert.out > run1.mira.out 2> mert.log

will hang with meteor-1.5.jar at around 13 iterations, but OK with meteor-1.4.jar.

kbmira is constantly waiting on the following line:

https://github.com/moses-smt/mosesdecoder/blob/master/mert/MeteorScorer.cpp#L140

The data to run the commands can be downloaded at

https://www.dropbox.com/s/2z5ne8fb4gt8exk/data.tgz?dl=0
mjdenkowski commented 9 years ago

If you uncomment the TRACE_ERR line, what input line is being sent to Meteor?

raingo commented 9 years ago

The last three lines:

in: EVAL |||60 117 13 65 3 3 9 9 0 0 0 0 1 1 0 0 5 5 0 0 18 18 18
out: 0.0925925925925926 
in: EVAL |||58 117 12 65 2 2 8 8 0 0 0 0 1 1 0 0 5 5 0 0 16 16 16
mjdenkowski commented 9 years ago

I was able to run your command with Meteor 1.5. It didn't hang but I did get an error I hadn't seen before and it exited with non-zero (edit: the error is from kbmira not Meteor):

Could not close file 4

It still wrote a seemingly correct mert.out so I don't know if this would actually cause problems with mert-moses.pl. Those input lines are also fine so I don't think the problem is with Meteor.

raingo commented 9 years ago

OK. I just realized the problem is that the meteor-1.5.jar I used is not the official one. I also got the error

Could not close file 4

, but the error is gone with the 3.0 release.