search

tensorflow / nmt

TensorFlow Neural Machine Translation Tutorial
Apache License 2.0
6.35k stars 1.96k forks source link

How to do Hyperparameter Optimization using Tensorflow NMT? #484

Open nashid opened 2 years ago

nashid commented 2 years ago

I need to try out different hyperparameters and compare their performance. I would be interested to know if an automated hyperparameter tuning option is available to do a Bayesian optimization as an example. However, it looks like the only option I have is to do a manual search.

To summarize what I want to know whether there are built-in methods in Tensorflow NMT to optimize hyperparameters. Unfortunately, I didn't find any. Ex: Grid search, Random Search.

maharajbrahma commented 2 years ago

@nashid I don't think this is an option for grid search or random search.

The current implementation is based on following papers.

  1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. ICLR.
  2. Minh-Thang Luong, Hieu Pham, and Christopher D Manning. 2015. Effective approaches to attention-based neural machine translation. EMNLP.
  3. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. NIPS