search

asyml / texar

Toolkit for Machine Learning, Natural Language Processing, and Text Generation, in TensorFlow. This is part of the CASL project: http://casl-project.ai/
https://asyml.io
Apache License 2.0
2.39k stars 371 forks source link

Gumbel softmax trick with beam search #209

Open Akella17 opened 4 years ago

Akella17 commented 4 years ago

Is it possible to use Gumbel softmax trick with a beam search decoder in the existing implementation of Texar. I want to differentiate through the generated beam search samples into the generator network.

ZhitingHu commented 4 years ago

1) The current beam-search implementation does not support customization with decoder Helpers, though I think it's possible to refactor beam-search to enable the nice feature.

2) Here is a nice paper on direct optimization of beam search with continuous approximation: https://arxiv.org/pdf/1708.00111.pdf

Any contributions to 1) or 2) are welcome!