search

google-research / albert

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Apache License 2.0
3.23k stars 571 forks source link

The output of the features of a given text consists of two tensor. Are they both the deep features? #240

Open FriedaSmith opened 3 years ago

FriedaSmith commented 3 years ago

I use the following code to get the features of a given text in PyTorch.

from transformers import AlbertTokenizer, AlbertModel

tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2')
model = AlbertModel.from_pretrained("albert-base-v2")
text = "Replace me by any text you'd like."
encoded_input = tokenizer(text, return_tensors='pt')
output = model(**encoded_input)

The result of output is shown in the figure.

output

output consists of two tensor. Are they both the deep features of a given text ?