executing mira.py: cat: write error: Broken pipe

[andidol]

I'm executing mira.py -d dev.de-en -c cdec.ini -w weights.init -j 8 -o mira-work --max-iterations 20 --optimizer 2 --metric-scale 1 -k 500 --update-size 500 --step-size 0.01 --hope 1 --fear 1 -g dev.de.g-o/grammar

• every file exists
• error occurs in line 372 mira.py

p1 = subprocess.Popen(['cat', source], stdout=subprocess.PIPE)

cat: write error: Broken pipe ERROR:root:Failed with exit code 1

[andidol]

After some debugging I found out that the error seems to occur during the following command:

/home/ubuntu/cdec/training/mira/../utils/parallelize.pl --use-fork -e /home/ubuntu/demo-en-de/mira-work/logs.0 -j 1 -- /home/ubuntu/cdec/training/mira/kbest_cut_mira -c cdec.ini -w /home/ubuntu/demo-en-de/mira-work/weights.0 -r/home/ubuntu/demo-en-de/mira-work/refs.input -m IBM_BLEU -s 1 -b 500 -k 500 -o 2 -p 00 -O /home/ubuntu/demo-en-de/mira-work/weights.pass0 -D /home/ubuntu/demo-en-de/mira-work -h 1 -f 1 -C 0.01 -u -e

**Environment: LOCAL /home/ubuntu/cdec/training/mira/../utils/parallelize.pl -> /home/ubuntu/cdec/training/mira/../utils Executing and gathering output: which /home/ubuntu/cdec/training/mira/kbest_cut_mira Executing and gathering output: basename /home/ubuntu/cdec/training/mira/kbest_cut_mira Parallelizing (1 ways): /home/ubuntu/cdec/training/mira/kbest_cut_mira -c cdec.ini -w /home/ubuntu/demo-en-de/mira-work/weights.0 -r/home/ubuntu/demo-en-de/mira-work/refs.input -m IBM_BLEU -s 1 -b 500 -k 500 -o 2 -p 00 -O /home/ubuntu/demo-en-de/mira-work/weights.pass0 -D /home/ubuntu/demo-en-de/mira-work -h 1 -f 1 -C 0.01 -u -e

-e dir: /home/ubuntu/demo-en-de/mira-work/logs.0 Executing and gathering output: dirname /home/ubuntu/cdec/training/mira/../utils/parallelize.pl Executing and gathering output: hostname Testing port 50399...port 50399 is available Running: /home/ubuntu/cdec/training/mira/../utils/sentserver -k 188092 50399
Executing: /home/ubuntu/cdec/training/mira/../utils/sentserver -k 188092 50399
Listening on port 50399

Client script:

cd /home/ubuntu /home/ubuntu/cdec/training/mira/../utils/sentclient ip-10-184-150-47:50399:188092 /home/ubuntu/cdec/training/mira/kbest_cut_mira -c cdec.ini -w /home/ubuntu/demo-en-de/mira-work/weights.0 -r/home/ubuntu/demo-en-de/mira-work/refs.input -m IBM_BLEU -s 1 -b 500 -k 500 -o 2 -p 00 -O /home/ubuntu/demo-en-de/mira-work/weights.pass0 -D /home/ubuntu/demo-en-de/mira-work -h 1 -f 1 -C 0.01 -u -e==== CHILD PROCESSES SPAWNED ... WAITING EXEC: /bin/bash -xeo pipefail ./workre0K.sh 1> /home/ubuntu/demo-en-de/mira-work/logs.0/kbes.1.OU 2> /home/ubuntu/demo-en-de/mira-work/logs.0/kbes.1.ER Executing and gathering output: /bin/bash -xeo pipefail ./workre0K.sh 1> /home/ubuntu/demo-en-de/mira-work/logs.0/kbes.1.OU 2> /home/ubuntu/demo-en-de/mira-work/logs.0/kbes.1.ER ERROR: Execution of /bin/bash -xeo pipefail ./workre0K.sh 1> /home/ubuntu/demo-en-de/mira-work/logs.0/kbes.1.OU 2> /home/ubuntu/demo-en-de/mira-work/logs.0/kbes.1.ER failed.

[JunjieHu]

I have the same problem of executing mira.py. I try cdec-2014-10-12 version provided on the cdec website. I get the same error on both mac and ubuntu 14.04.

./training/mira/mira.py -d dev.lc-tok.es-en.sgm -t devtest.lc-tok.es-en.sgm -c cdec.ini -j 1 WARNING:root:Failed to import matplotlib, graphs will not be generated. INFO:root:WORKING DIRECTORY=/home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019 INFO:root:INI FILE=cdec.ini INFO:root:DEVSET=/home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019/dev.input INFO:root:EVAL METRIC=IBM_BLEU INFO:root:MAX ITERATIONS=20 INFO:root:PARALLEL JOBS=1 INFO:root:INITIAL WEIGHTS=/home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019/weights.0 INFO:root:TEST SET=devtest.lc-tok.es-en.sgm INFO:root:using port 33416 INFO:root:======= STARTING ITERATION 0 ======= INFO:root:Starting at Sat May 28 11:20:19 2016 INFO:root: log directory=/home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019/logs.0 INFO:root:OPTIMIZATION COMMAND: /home/jjhu/Research/cdec-2014-10-12/training/mira/../utils/parallelize.pl --use-fork -e /home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019/logs.0 -j 1 -- /home/jjhu/Research/cdec-2014-10-12/training/mira/kbest_cut_mira -c cdec.ini -w /home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019/weights.0 -r/home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019/refs.input -m IBM_BLEU -s 1 -b 500 -k 500 -o 2 -p 00 -O /home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019/weights.pass0 -D /home/jjhu/Research/cdec-2014-10-12/mira.dev.lc-tok.es-en.20160528-112019 -h 1 -f 1 -C 0.001 -u -e ERROR:root:Failed with exit code 1

Can you help to fix it? @redpony @vchahun