Least invasive way to use a decoder-only Transformer with fast decoding?

tensorflow / tensor2tensor

Library of deep learning models and datasets designed to make deep learning more accessible and accelerate ML research.

Apache License 2.0

15.6k stars 3.51k forks source link

Least invasive way to use a decoder-only Transformer with fast decoding? #493

Closed AranKomat closed 6 years ago

AranKomat commented 6 years ago

I'd like to use a decoder-only Transformer with fast coding ability to use it as a RL agent in NLP like SeqGAN. transformer_moe.py has decoder-only option, but it seems to require onerous modification of fast decoding method and its related methods of transformer.py. What's your thought?

rsepassi commented 6 years ago

There's a TransformerDecoder model that you may be able to use. However, you may still have to modify the fast-decode codepath to have it work.

AranKomat commented 6 years ago

Thanks so much for your advice. It works very well.

phiresky commented 5 years ago

Looks like TransformerDecoder was removed in https://github.com/tensorflow/tensor2tensor/commit/5aedc3deda7b5e640f201874c38413822cb4daf3 by you, @rsepassi . Is this still possible somehow? My goal is to train a transformer to decode sentence embeddings.

tensorator commented 5 years ago

I have the same exact requirement, any luck @phiresky ?