-
# ❓ Questions & Help
## Details
As far as I can tell, when using BERT with cross attention and output_attention is True, the returned attention only contains self attention (i.e. a tuple of …
-
# ❓ Questions & Help
I have been attempting with various models to try to build an encoder-decoder, sequence to sequence transformer model. For the most part, I have been using BERT (bert-base-case…
-
# ❓ Questions & Help
Hi, everyone. I need help with the encoding-decoding model. I'm trying to train the model to create a title for a small text.
I'm creating a basic Encode-Decode model with B…
-
# ❓ Questions & Help
## Details
Hey , I want to ask the following questions.
How is the loss calculated in DecoderEncoderModel. What is the mathematic formula of the loss function ?
I jus…
AI678 updated
4 years ago
-
# Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [x] I am using the latest TensorFlow Model Garden release and TensorFlow 2.
- [x] I am reporting…
-
**Describe the bug**
Hello,
When I train Encoder Decoder seq2seq, I get an error. Error message is
WARNING:transformers.tokenization_utils_base:Truncation was not explicitely activated but `max_…
-
# ❓ Questions & Help
## Details
I'm recently building a encoder-decoder model (Bert2Bert) using `encoderdecodermodel`.
But I found that it is really hard to get cross attention weights …
-
# ❓ Questions & Help
## Details
Hey , I use EncoderDecoderModel for abstractive summarization. I load the bert2bert model like this
model=EncoderDecoderModel.from_encoder_decoder_pretraine…
yhznb updated
4 years ago
-
# ❓ Questions & Help
## Details
I follow the EncoderDecoder tutorials(https://huggingface.co/transformers/model_doc/encoderdecoder.html), but it seems that when i use generate function,
ge…
-
I try to use EncoderDecoderModel to train a Chinese summary model.
```Python
from transformers import BertConfig, EncoderDecoderConfig, EncoderDecoderModel
encoder_config = BertConfig.from_pre…