BrokenPipeError: [Errno 32] Broken pipe

[zmykevin]

Hi, I encountered a issue when I use nmtpy to train a model. Whenever it comes to the step to perform evaluation, it will always generate a BrokenPipe Error. Traceback (most recent call last):

File "/home/zmykevin/software/miniconda2/envs/nmtpy_pytorch/bin/nmtpy", line 6, in exec(compile(open(file).read(), file, 'exec')) File "/home/zmykevin/machine_translation_vision/code/mtv_kevin/nmtpytorch/bin/nmtpy", line 132, in loop() File "/home/zmykevin/machine_translation_vision/code/mtv_kevin/nmtpytorch/nmtpytorch/mainloop.py", line 246, in call while self.train_epoch(): File "/home/zmykevin/machine_translation_vision/code/mtv_kevin/nmtpytorch/nmtpytorch/mainloop.py", line 146, in train_epoch self.do_validation() File "/home/zmykevin/machine_translation_vision/code/mtv_kevin/nmtpytorch/nmtpytorch/mainloop.py", line 217, in do_validation results.extend(self.evaluator.score(hyps)) File "/home/zmykevin/machine_translation_vision/code/mtv_kevin/nmtpytorch/nmtpytorch/evaluator.py", line 38, in score scorer.compute(self.refs, hyps, **self.kwargs[key])) File "/home/zmykevin/machine_translation_vision/code/mtv_kevin/nmtpytorch/nmtpytorch/metrics/meteor.py", line 55, in compute proc.stdin.write(line + '\n') BrokenPipeError: [Errno 32] Broken pipe

I wonder if you have any idea what causes this problem? Thanks.

[ozancaglayan]

Hello,

Do you have java installed?

[zmykevin]

I figured out the solution. I have another question. Is the cGRU model you provided in this repository the same as the one for your work on LIUMCVC model that is submitted to WMT 2017? Can I reproduce the results of that model on Multi30K 2017 test dataset by using nmtpy to train this model? I have tried to train it for once, but only get BLEU score of 23.69 on the test dataset. Thank you for your time to answer this question.

[ozancaglayan]

This repository is a complete re-iteration of the previous nmtpy-theano so I wouldn't expect to see exactly the same results. For now, I did not implement the multimodal systems of WMT2017 but I trained many systems with the monomodal nmt and obtain BLEU and METEOR which are on-par with last year.

[zmykevin]

Hi Orzan, Thank you for replying to my previous message. Do you mind to share some suggestions on how to finetune the hyperparameters in order to achieve an on-par performance as reported in the WMT2017 Multimodal Machine Translation report?

For now, I have used the default setting defined in "examples/multi30k-en-de-im-bpe10k.conf". I checked the hyperparameters in this configuration file and also go through the code that you write to define the NMT.py model, I think this should be pretty similar to the text-only NMT structure that you have proposed in the WMT 17 MMT challenge. A couple of key things that I noticed in this configuration file is that: (1) dropout_emb = 0.3 (2) dropout_ctx = 0.5 (3) dropout_out = 0.5 (4) emb_dim = 128 (5) enc_dim = 256 (6) dec_dim = 256 (7) lr = 0.0004 These parameter settings are matching what you have reported in the paper. I wonder if I want to achieve a better performance, what parameters should I spend more time to finetune? Or there needs some change in the network structure?

Another question that I have is that you claimed in the paper that after conducting ensembling on 5 different models trained on 5 runs you get the final result which gives you another 1 percent improvement on BLEU score. I wonder how do you exactly conduct the ensembling? Is there a resource that you can point me to so that I can also implement this process?

Thank you very much for spending time answering my questions.

[ozancaglayan]

Hello,

The ensembling is not added yet into this repository. As I told before, MMT17 systems were trained with the old nmtpy which was based on Theano so I won't expect to see exactly the same results.

As per the parameter to tune, well that's a whole other story that people attack by doing hyperparameter search.