search

stanford-crfm / BioMedLM

590 stars 61 forks source link

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. #25

Open zhengbiqing opened 10 months ago

zhengbiqing commented 10 months ago

I run the "Example Usage":

import torch

from transformers import GPT2LMHeadModel, GPT2Tokenizer

device = torch.device("cuda")

tokenizer = GPT2Tokenizer.from_pretrained("stanford-crfm/BioMedLM")

model = GPT2LMHeadModel.from_pretrained("stanford-crfm/BioMedLM").to(device)

input_ids = tokenizer.encode( "Photosynthesis is ", return_tensors="pt" ).to(device)

sample_output = model.generate(input_ids, do_sample=True, max_length=50, top_k=50)

print("Output:\n" + 100 * "-") print(tokenizer.decode(sample_output[0], skip_special_tokens=True))

zhengbiqing commented 10 months ago

I got the result：

The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results. Setting pad_token_id to eos_token_id:28895 for open-end generation. Output:

Photosynthesis is [photosynthetic activity]... [that] is one of the fundamental capabilities of plants on Earth. I would never be thinking about that in light of a world without oxygen. There are still oxygen

what‘s the problem? Anyone can help?

zhengbiqing commented 10 months ago

The "stanford-crfm/BioMedLM" in code is replaced by "/root/siton-glusterfs-eaxtsxdfs/xts/data/BioMedLM" where I download the model.