search

tensorflow / nmt

TensorFlow Neural Machine Translation Tutorial
Apache License 2.0
6.39k stars 1.96k forks source link

when i use large dev set like 100,000 size, the decoding step when training is very slow, takes about 2 hours once #128

Open zhangpengGenedock opened 7 years ago

zhangpengGenedock commented 7 years ago

Can someone help me speed up?

zhangpengGenedock commented 7 years ago

{ "attention": "normed_bahdanau", "attention_architecture": "gnmt_v2", "batch_size": 64, "bpe_delimiter": "@@", "colocate_gradients_with_ops": true, "decay_factor": 0.5, "decay_steps": 17000, "dropout": 0.2, "encoder_type": "gnmt", "eos": "", "forget_bias": 1.0, "infer_batch_size": 32, "init_weight": 0.1, "learning_rate": 1.0, "max_gradient_norm": 5.0, "metrics": ["bleu"], "num_buckets": 5, "num_layers": 8, "num_train_steps": 340000, "num_units": 1024, "optimizer": "sgd", "residual": true, "share_vocab": false, "sos": "", "source_reverse": false, "src_max_len": 50, "src_max_len_infer": null, "start_decay_step": 170000, "steps_per_external_eval": null, "steps_per_stats": 100, "tgt_max_len": 50, "tgt_max_len_infer": null, "time_major": true, "unit_type": "lstm", "beam_width": 10, "length_penalty_weight": 0.0 }

zhenbiaochen commented 7 years ago

set the dev set 1000 sentences

oahziur commented 7 years ago

@zhangpengGenedock Alternatively, you can disable the beam search if you really want to go over the entire dev set during training (set beam_width to 0)