Why there is not any usage example

[Oxi84]

I appreciate you hard work on the language models, but I do not really understand why there is not any code example how to predict a masked word or hidden states at least using pretrained models.

Lets say that I have a sentence such as "This is my sentence".
How do I predict distribution probability for word .

pseudocode:

  text = "This is my <mask> sentence". 
  tokens = tokenize(text)
 masked_index = 3 # for example
 model = something.load(modelfilename)
 predictions = model(tokens)
 logits_masked_index = predictions[masked_index]

[hiwaveSupport]

yes this is sorely missing - just simple example using pre-trained models.