[Blender] How to calculate loss from Blender?

[josharnoldjosh]

Hello,

I have a subclassed blender agent:

class Blender(TransformerGeneratorAgent):

I first generated a bunch of sentences: ["hey, how are you?", "how's it going..?"]

How can I calculate the loss from this subclassed agent?

Would it be something like:

blender = Blender()
loss = blender.calculate_loss(sentences, ground_truth_sentences)

My end goal is to calculate the KL Divergence between two similar losses outputted from Blender models, and then eventually call loss.backward() (Trying to figure out how to apply Proximal Policy Optimization).

Small other question: How to load the pre-trained blender tokenizer? Also, I apologize if this is a very novice question, but how do I retrieve the tokenizer that the pre-trained blender model used to tokenize sentences? Do I initialize a tokenizer type class and somehow load its information from the data/models/blender_90M/ file? I am calling blender.act() which returns directly the text, and not the tokenized or encoded sentences?

Thank you so much for all the help! I am a bit of a novice with this type work :)

[stephenroller]

Ugh did I not reply to this?

• If you give set the opt['verbose'] flag to True, the per-token loss will be returned as part of the Message. This is useful if you want to do some sort of RL
• If you extend the agent, you can override compute_loss, which is where the loss is computed.
• After you load the agent, the tokenizer is available in self.dict, particularly the self.dict.txt2vec method.

[josharnoldjosh]

Thank you!!!