search

tensorflow / nmt

TensorFlow Neural Machine Translation Tutorial
Apache License 2.0
6.38k stars 1.96k forks source link

decoding to output (dev/test) - 98% of CPU loading? #110

Open myurushkin opened 7 years ago

myurushkin commented 7 years ago

Hi there!

My GPU: gtx1070, TF 1.3

Is it ok that evaluation of test/dev datasets is performed on CPU mostly? GPU loading: 20% CPU loading: >90%

In my case training doesn't take much time in comparision with evaluation. Is it ok?

Best regards, Mikhail

oahziur commented 7 years ago

@myurushkin Are you using length penalty weight > 0.0 for beam search? You may disable it in your training for now as we found it doesn't utilize GPU well. A fix will be pushed to the future version of tensorflow.

myurushkin commented 7 years ago

Yep, I have used one of the default configurations: wmt16_gnmt_4_layer.json length_penalty_weight=1 in this case. I'll set this parameter to 0. Thanks!

okuchaiev commented 6 years ago

@oahziur is there a TF issue to track this? Thanks!

oahziur commented 6 years ago

@okuchaiev This should be fixed in tensorflow version 1.4