loss

[xuewyang]

Hi, I am using sum of two losses as the final loss to optimize. Is there a way to use one of them as the loss for saving the best model? Not the sum. Thank you.

[alasdairtran]

Are you still using my CallbackApexTrainer? If so in the config file, you can add an extra field called validation_metric:

trainer:
  type: callback_apex
  ...
  callbacks:
    ...
    - type: track_metrics
      validation_metric: "-loss"

By default it is -loss, so we save the model with the lowest loss (the highest -loss). You can change it to something else like -custom_loss_1. Just make sure the get_metrics() method in your Model class returns an output dict that contains a key custom_loss_1. See here for how I implemented mine. Or here's a simple example:

def get_metrics(self, reset: bool = False) -> Dict[str, float]:
    metrics = {
        'custom_loss_1': self.total_custom_loss_1 / self.n_batches,
    }
    if reset: # is True at the end of each training epoch and each validation round
        self.total_custom_loss_1 = 0
        self.n_batches = 0
    return metrics

You will need to manually keep a running total of self.total_custom_loss_1 (as plain Python floats) and self.n_batches in your forward function.