Closed woctezuma closed 4 years ago
tl;dr: my questions are:
_ce
in train_loss_ce
stand for? It should be the classification loss, so it might be ce for "classfication error"?_unscaled
metrics?On a side note, I have not delved too deep into the panoptic segmentation part of the paper.
Are the following losses mentioned in main.py
the coefficients for the Focal loss and the Dice loss respectively, as mentioned in section 4.4 on page 15?
# * Loss coefficients
parser.add_argument('--mask_loss_coef', default=1, type=float)
parser.add_argument('--dice_loss_coef', default=1, type=float)
Info is all in: https://github.com/facebookresearch/detr/blob/master/models/detr.py
def loss_boxes(self, outputs, targets, indices, num_boxes):
"""Compute the losses related to the bounding boxes, the L1 regression loss and the GIoU loss
targets dicts must contain the key "boxes" containing a tensor of dim [nb_target_boxes, 4]
The target boxes are expected in format (center_x, center_y, w, h), normalized by the image size.
"""
Answer about the _ce
suffix: it is for cross-entropy. That is the loss about classes.
def loss_labels(self, outputs, targets, indices, num_boxes, log=True):
"""Classification loss (NLL)
targets dicts must contain the key "labels" containing a tensor of dim [nb_target_boxes]
"""
[...]
loss_ce = F.cross_entropy(src_logits.transpose(1, 2), target_classes, self.empty_weight)
losses = {'loss_ce': loss_ce}
Answer about class error:
if log:
# TODO this should probably be a separate loss, not hacked in this one here
losses['class_error'] = 100 - accuracy(src_logits[idx], target_classes_o)[0]
return losses
where:
def accuracy(output, target, topk=(1,)):
"""Computes the precision@k for the specified values of k"""
if target.numel() == 0:
return [torch.zeros([], device=output.device)]
maxk = max(topk)
batch_size = target.size(0)
_, pred = output.topk(maxk, 1, True, True)
pred = pred.t()
correct = pred.eq(target.view(1, -1).expand_as(pred))
res = []
for k in topk:
correct_k = correct[:k].view(-1).float().sum(0)
res.append(correct_k.mul_(100.0 / batch_size))
return res
Answer about cardinality error:
def loss_cardinality(self, outputs, targets, indices, num_boxes):
""" Compute the cardinality error, ie the absolute error in the number of predicted non-empty boxes
This is not really a loss, it is intended for logging purposes only. It doesn't propagate gradients
"""
Answer about "mask" (Focal) loss and "Dice loss:
def loss_masks(self, outputs, targets, indices, num_boxes):
"""Compute the losses related to the masks: the focal loss and the dice loss.
targets dicts must contain the key "masks" containing a tensor of dim [nb_target_boxes, h, w]
"""
[...]
losses = {
"loss_mask": sigmoid_focal_loss(src_masks, target_masks, num_boxes),
"loss_dice": dice_loss(src_masks, target_masks, num_boxes),
}
Hey, sorry for not getting back to you before.
Your findings are correct. One last point about the _unnormalized
losses: we have scaling coefficients for each loss that you can see in https://github.com/facebookresearch/detr/blob/5e66b4cd15b2b182da347103dd16578d28b49d69/main.py#L74-L77
those scaling coefficients are used to balance the contribution of each loss to the total loss. The _unscaled
values you see in the logs are the original values of the losses, before being scaled by those scaling coefficients, as you can see in https://github.com/facebookresearch/detr/blob/5e66b4cd15b2b182da347103dd16578d28b49d69/engine.py#L39-L42
Thank you. It makes sense, and it answers one of the questions which I had (but forgot to ask in my long writing).
In
log.txt
, I see several errors and losses. I would like to know a bit more about some of them.I would expect the log file to contain:
train_loss
),train_loss_ce
,train_loss_bbox
,train_loss_giou
), as detailed in section 3.1 of the paper with the Hungarian loss, then table 4, and also on page 12, where it is mentioned that:1) In the log file, first, there are these:
First, it is mostly fine, except I am not sure what
ce
stands for. I assume it is the loss associated with the classification error. Is that correct?Second, I would assume that "class error" is the "weighted fraction of misclassified observations". However, I have seen cases where it was much higher that 1. Isn't it supposed to be between 0 and 1?
2) Then, with the suffix going from 0 to 4:
This looks OK.
I assume these are layer-specific losses, which allow to compute the Hungarian loss as mentioned in section Auxiliary decoding losses in the paper:
This is consistent with the fact that there are 6 decoding layers by default:
3) Finally:
I don't know what "cardinality error" is.
I don't know why
train_class_error_unscaled
is mentioned, because the classification error is not going to be normalized like the losses are.Apart from that, it looks OK, as I assume the suffix
_unscaled
means before the normalization mentioned in appendix A.2: