CAIC-AD / YOLOPv2

YOLOPv2: Better, Faster, Stronger for Panoptic driving Perception
MIT License
544 stars 67 forks source link

Object detection class only has car? #53

Open Bridgee opened 10 months ago

Bridgee commented 10 months ago

Based on the label format of BDD100k, there should be 10 classes annotated. I have noticed that according to the label format of BDD100k, there should be ten annotated classes. However, when using the pre-trained model, all the output is classified as class 3, which is "car". I am wondering if the model is designed to only output car, as I cannot find any relevant information in the paper.

harrylal commented 9 months ago

Seems like the model might be lumping car, truck, bus, and train into one category named "vehicles," and train for one class. similar to what other models within the same family(hybridnets, yolop etc) do for fair comparisons. But the ID 3 indeed feels odd.

Also myself and @nikkita-28, We took a peek at the code and spotted something interesting: the detection head's output tensor shows confidences for a whopping 80 classes(coco?) and the max out of that always comes out to be at index 3!

if multi_label:
      i, j = (x[:, 5:] > conf_thres).nonzero(as_tuple=False).T
      x = torch.cat((box[i], x[i, j + 5, None], j[:, None].float()), 1)
else:  # best class only
      conf, j = x[:, 5:].max(1, keepdim=True)

Any chance the devs could shed some light on this? Thanks a bunch!