Modifying gt_tensor

[YashRunwal]

@yjh0410 Hi,

I am using this CenterNet model for my dataset. I have modified a few things to make it work for me and I can train for Object Detection. However, I am trying to predict another value using the modified CenterNet ( I am concatenating 2 different types of images).

So my dataset annotation looks like this: x1, y1, x2, y2, class_id, d_val d_val ranges from 0 to 1.

Currently, the gt_tensorhas the shape [128, 384, 17]. This is excluding the d_val from the annotations.

I understand that if I want to include the d_val inside the gt_tensor, the shape of the gt_tensorwill be [128, 384, 18]. Like: gt_tensor = np.zeros([hs, ws, num_classes + 4 + 1 + 4 + 1]) -> gt_tensor: [128, 384, 18].

But how do I fill up the d_val inside the gt_tensor? The d_val is associated with each bounding box, i.e. Each bounding box in the annotations will have a d_val value.

I would really appreciate your help.

[YashRunwal]

@yjh0410 So I changed the gt_tensor script as follows:

gt_tensor = np.zeros([hs, ws, num_classes + 4 + 1 + 4 + 1])
    gt = []

    for gt_label in range(len(label_lists)):

        # cls_id = int(label_lists[gt_label][-1])
        ################# NEW ###################
        cls_id = int(label_lists[gt_label][-2])

        result = generate_txtytwth(label_lists[gt_label], w, h, s)
        if result:
            grid_x, grid_y, tx, ty, tw, th, weight, rw, rh, x1, y1, x2, y2 = result

            gt_tensor[grid_y, grid_x, cls_id] = 1.0
            gt_tensor[grid_y, grid_x, num_classes:num_classes + 4] = np.array([tx, ty, tw, th])
            gt_tensor[grid_y, grid_x, num_classes + 4] = weight
            # gt_tensor[grid_y, grid_x, num_classes + 5:] = np.array([x1, y1, x2, y2])
            gt_tensor[grid_y, grid_x, num_classes + 5:-1] = np.array([x1, y1, x2, y2])

            # Adding depth val ################## NEW ##################
            gt_tensor[grid_y, grid_x, -1] = label_lists[gt_label][-1]

As You see, I added another value to the gt_tensor. I would like to know if what I've done is correct or not? So this depth value is for each bounding box and each class.

I desperately need your help.

[YashRunwal]

@yjh0410 Sorry for tagging you here again. But what do you think of the aforementioned addition to the gt_tensor. I trained the model and the depth value should be ranging between 0 and 200, however, the results aren't great. I did add a depth prediction head and a SmoothL1Loss for this too. What do you think?