Conditional generation with T5

[ShivanshuPurohit]

Environment info

• transformers version: 4.3.2
• Platform: Linux-4.19.112+-x86_64-with-Ubuntu-18.04-bionic
• Python version: 3.6.9
• PyTorch version (GPU?): 1.7.0+cu101 (True)
• Tensorflow version (GPU?): 2.4.1 (True)
• Using GPU in script?: yes
• Using distributed or parallel set-up in script?: no

Who can help

Information

Model I am using (Bert, XLNet ...): T5

The problem arises when using:

• [ ] the official example scripts: (give details below)
• Generating conditional text from T5

  from transformers import T5Tokenizer, T5ForConditionalGeneration
  tokenizer = T5Tokenizer.from_pretrained('t5-3b')
  model = T5ForConditionalGeneration.from_pretrained('t5-3b')
  input_ids = tokenizer('The <extra_id_0> walks in <extra_id_1> park', return_tensors='pt').input_ids
  labels = tokenizer('<extra_id_0> cute dog <extra_id_1> the <extra_id_2> </s>', return_tensors='pt').input_ids
  outputs = model(input_ids=input_ids, labels=labels)
  loss = outputs.loss
  logits = outputs.logits
  input_ids = tokenizer("summarize: studies have shown that owning a dog is good for you ", return_tensors="pt").input_ids  # Batch size 1
  outputs = model.generate(input_ids)

To reproduce

Steps to reproduce the behavior:

1. Run the code above

Expected behavior

To see the generated text. Rather, the model outputs a torch Tensor like so tensor([[ 0, 363, 19, 8, 1784, 13, 1473, 58, 1]]) How do I get words out of it rather than a tensor?

[NielsRogge]

You can decode them back to a string using T5Tokenizer, like so:

tokenizer.decode(outputs.squeeze().tolist(), skip_special_tokens=True)

Btw, for a really good guide on the different generation strategies of models like T5, see this blog post: https://huggingface.co/blog/how-to-generate

[ShivanshuPurohit]

This post was really helpful, thanks!