MODIFY yolov8-pose.py file

[xcn700418]

Hi, I want to use customized yolov8-pose for auto-labeling but I encountered some issue. The original yolov8-pose model has output with dimension 56 which corresponds to 4 bboxes coordinate, 1 bbox confidence score, and 172 keypoint coodinates and 171 keypoint score. For my customized yolov8-pose model, my output has dimension 13 which corresponds to 4 bboxes coordinate, 1 bbox confidence score, and 4*2 keypoint coodinates.

Without modifying the anylabeling\services\auto_labeling\yolov8-pose.py script, the error shows "Error in predict_shapes: not enough values to unpack (expected 3, got 2)", which completely make sense because I only got 4*2 dimensions for keypoint without the keypoint score. So I comment the original python code and modify the python code as shown below,

interval = 3

        # for i in range(0, len(kpts), interval):
        #     x, y, kpt_score = kpts[i : i + 3]
        #     inside_flag = point_in_bbox((x, y), xyxy)
        #     if (kpt_score > self.conf_thres) and inside_flag:
        #         label = self.keypoints[int(i // interval)]
        #         point_shape = Shape(
        #             label=label, shape_type="point", group_id=group_id
        #         )
        #         point_shape.add_point(QtCore.QPointF(x, y))
        #         shapes.append(point_shape)
        interval = 2
        for i in range(0, len(kpts), interval):
            x, y = kpts[i : i + 2]
            inside_flag = point_in_bbox((x, y), xyxy)
            if inside_flag:
                label = self.keypoints[int(i // interval)]
                point_shape = Shape(
                    label=label, shape_type="point", group_id=group_id
                )
                point_shape.add_point(QtCore.QPointF(x, y))
                shapes.append(point_shape)

However, a new error occured as shown below, "
lambda auto_labeling_result: self.parent.new_shapes_from_auto_labeling( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\lenovo\Desktop\annotations\X-AnyLabeling\anylabeling\views\labeling\label_widget.py", line 4784, in new_shapes_from_auto_labeling self.load_shapes(auto_labeling_result.shapes, replace=True) File "C:\Users\lenovo\Desktop\annotations\X-AnyLabeling\anylabeling\views\labeling\label_widget.py", line 2581, in load_shapes self.add_label(shape) File "C:\Users\lenovo\Desktop\annotations\X-AnyLabeling\anylabeling\views\labeling\label_widget.py", line 2513, in add_label self.unique_label_list.set_item_label( File "C:\Users\lenovo\Desktop\annotations\X-AnyLabeling\anylabeling\views\labeling\widgets\unique_label_qlist_widget.py", line 36, in set_item_label qlabel.setText("{}".format(html.escape(label))) ^^^^^^^^^^^^^^^^^^ File "C:\Users\lenovo\AppData\Local\Programs\Python\Python312\Lib\html__init__.py", line 19, in escape s = s.replace("&", "&") # Must be done first! ^^^^^^^^^ AttributeError: 'int' object has no attribute 'replace' "

May I ask how to make correct modifications in this case?

[CVHub520]

Hi, @xcn700418:

Thank you for reaching out. Now, let's look at the error message:

AttributeError: 'int' object has no attribute 'replace'"

This error indicates that you're trying to use the replace method on an integer object, which is not allowed in Python. This is likely because the label variable is an integer and you're trying to set it as a label in the GUI, which requires a string.

To resolve this, you need to convert the label to a string before setting it. Here's how you can do it:

...
point_shape = Shape(
    label=str(xxx),  # Convert the label to a string
    shape_type="point",
    group_id=group_id
)
...

By converting the label to a string, you ensure that it is treated as a label and not as an integer when you try to set it in the GUI.

If you continue to face issues, please provide more details about the code and the exact error messages you’re encountering. This will help in providing a more accurate solution.

Best regards, CVHub

[xcn700418]

Thanks so much for your reply, the issue has been solved after adding str().

A new issue occured, the output dimension of my model is 1138400, where 13 corresponds to 4 bboxes coordinate, 1 bbox confidence score, and 4 keypoint coodinates as I mentioned previously. However, after auto-labeling the images, there are only 1 or 2 keypoints that have been successfully predicted and labeled, and I am pretty sure it is not related to my model since I have already use some test images for model inference, the keypoints can be correctly predicted for most of the images.

I noticed that there is an if statement to check whether the predicted keypoint is in the bbox using variable "inside_flag", so I think some keypoints might be outside the bbox, so I tried to get rid off the if statement to label all the predicted keypoint as shown below,

        interval = 2
        for i in range(0, len(kpts), interval):
            x, y = kpts[i : i + 2]
            inside_flag = point_in_bbox((x, y), xyxy)
            label = self.keypoints[int(i // interval)]
            point_shape = Shape(
                label=str(label), shape_type="point", group_id=group_id
            )
            point_shape.add_point(QtCore.QPointF(x, y))
            shapes.append(point_shape)

After modifying the code and re-labeling the image, I found that for most of the imgaes, there are always 2 points outside the bbox, and I don't know why. Do you have any suggestion?

[xcn700418]

As shown in the images above, the red and green point can always be predicted correctly, but the blue and purple point are always predicted wrong in same relative position.

[CVHub520]

Hello, I strongly recommend that you first write an inference script based on ORT to ensure everything is working properly before migrating to X-AnyLabeling.