TaskAlignedAssigner doesn't take into account `reg_max` which affects prediction of large objects

[Y-T-G]

Search before asking

• [X] I have searched the YOLOv8 issues and found no similar bug report.

YOLOv8 Component

Train

Bug

The TaskAlignedAssigner is responsible for choosing best anchor point that best fits the target bounding box. But the assignment does not take into account the reg_max property of the head. That means select_candidates_in_gts will choose anchor points that fits the center point of the target bounding box but due to the reg_max limit of 16 which it is not taking into account, it may be using the anchor points that is coming from an FPN layer that can't entirely cover the whole box due to reg_max. For example, choosing an anchor point from the 20x20 feature maps would be suitable for an object that covers the whole image. But if TAL assigns an anchor point from the 40x40 or 80x80 feature maps, the predicted box will not be able to cover the whole box as reg_max limits the regression to encompass just 30x30 portion of the feature maps.

This problem is evident across multiple users reporting that the predictions are cropped for large objects.

https://github.com/ultralytics/ultralytics/issues/6777 https://github.com/ultralytics/ultralytics/issues/6062 https://github.com/ultralytics/ultralytics/issues/4961 https://github.com/ultralytics/ultralytics/issues/4988#issuecomment-1729252938 https://github.com/ultralytics/ultralytics/issues/4973 https://github.com/ultralytics/ultralytics/issues/2949

Environment

No response

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• [X] Yes I'd like to help by submitting a PR!

[glenn-jocher]

@Y-T-G thank you for pointing out the issue with how TaskAlignedAssigner handles reg_max when predicting large objects. This definitely seems to be a critical aspect not currently accounted for, affecting the predictions for larger objects as you described.

We appreciate your willingness to submit a PR! It would be fantastic to see your approach to adjusting the TaskAlignedAssigner to consider reg_max. If you need any guidance or assistance while preparing the PR or if you would like to discuss potential solutions, don't hesitate to reach out. 🚀

Looking forward to your contribution!

[Burhan-Q]

@Laughing-q do have any thoughts on this? I followed a Discord discussion that mentions this issue and I've seen at least one other case where this may have been happening as well.