Accessing scores for the entire vocabulary in GPT2

[shampp]

I want to access the prediciton scores from the GPT2 model.

Follwoing the example given in the docstring, I am using the code

    output = model(input_ids,labels=input_ids)
    logit = output[1][0][-1,:].detach().numpy()

Is it correct ? I am expecting logit variable to be the same size as the vocabulary with corresponding scores.

[LysandreJik]

You would need to use GPT2LMHeadModel for that, and use a softmax layer to have scores. Here's an example:

from transformers import GPT2LMHeadModel, GPT2Tokenizer
import torch

model = GPT2LMHeadModel.from_pretrained("gpt2")
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")

inputs = tokenizer.encode("This is just", return_tensors="pt")
output = model(inputs)

model_output = output[0]
last_token_prediction = model_output[:, -1]
last_token_softmax = torch.softmax(last_token_prediction, dim=-1).squeeze()

n = 10
top_n_values = last_token_softmax.topk(n)

for index, value in zip(top_n_values.indices, top_n_values.values):
    print("Score: ", value.tolist())
    print("This is just" + tokenizer.decode(index.tolist()))