Open bookpen opened 11 months ago
I find there is no softmax function when I should get the distribution of prediction.
wrapper.py def mlm_train_step(self, labeled_batch: Dict[str, torch.Tensor], unlabeled_batch: Optional[Dict[str, torch.Tensor]] = None, lmtraining: bool = False, alpha: float = 0, **) -> torch.Tensor: """Perform a MLM training step."""
inputs = self.generate_default_inputs(labeled_batch) mlm_labels, labels = labeled_batch['mlm_labels'], labeled_batch['labels'] outputs = self.model(**inputs) prediction_scores = self.preprocessor.pvp.convert_mlm_logits_to_cls_logits(mlm_labels, outputs[0]) loss = nn.CrossEntropyLoss()(prediction_scores.view(-1, len(self.config.label_list)), labels.view(-1))
the prediction_scores is not applied to the softmax
I find there is no softmax function when I should get the distribution of prediction.
wrapper.py def mlm_train_step(self, labeled_batch: Dict[str, torch.Tensor], unlabeled_batch: Optional[Dict[str, torch.Tensor]] = None, lmtraining: bool = False, alpha: float = 0, **) -> torch.Tensor: """Perform a MLM training step."""
the prediction_scores is not applied to the softmax