search

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
https://huggingface.co/transformers
Apache License 2.0
134.79k stars 26.96k forks source link

BERT as encoder and a transformer as a decoder. #4111

Closed ayanamongol closed 4 years ago

ayanamongol commented 4 years ago

Hi, there Is there probability to build a model of the BERT as the encoder and the transformer as the decoder?

Thanks.

bugface commented 4 years ago

The encoder-decoder or bart may be what you want.

patrickvonplaten commented 4 years ago

I think you should take a look into the encoder-decoder framework: https://huggingface.co/transformers/model_doc/encoderdecoder.html

patrickvonplaten commented 4 years ago

Note that currently only Bert2Bert is possible.