Addressing Low Assigned Scores and Weighting Issues in Key Point Detection Training with YOLO-NAS-POSE:

[ZYX-MLer]

💡 Your Question

I am training a keypoint detection algorithm using YOLO-NAS-POSE and observed that the training loss is very low, yet there is no improvement in the model's performance. During the debugging process, I found that in the early stages of training, the values of assign_result.assigned_scores are extremely small. Various losses weighted by assigned_scores are also very small, which preventing effective model optimization.

I noticed the following statement during the weighting process: assigned_scores_sum = torch.clip(assigned_scores_sum, min=1.0), which future prevents very small weights from being amplified.

Am I correct in my understanding? How should I modify it to improve training performance?

Thank you.

Versions

No response

[BloodAxe]

At the beginning of training when detection part is weak and provides poor detection boxes a keypoint part will also lag behind. This is sort of expected and the solution is to just train it longer.

You can set rescale_pose_loss_with_assigned_score: False and assigner_multiply_by_pose_oks: False for YoloNASPoseLoss to disable this, but we empirically found that having those values of True gives higher final AP score, while training may take longer to converge.