ultralytics / ultralytics

Ultralytics YOLO11 πŸš€
https://docs.ultralytics.com
GNU Affero General Public License v3.0
36.33k stars 7k forks source link

Enhance `classify` results plotting #18978

Open RizwanMunawar opened 1 week ago

RizwanMunawar commented 1 week ago

@glenn-jocher Hi, this PR enhances the visualization of classification results. Currently, there are no background rectangles or distinct text colours for each classified object. This PR introduces these features. Thanks

Sample# Old version This PR
01 Bus 1 Bus 2

πŸ› οΈ PR Summary

Made with ❀️ by Ultralytics Actions

🌟 Summary

Enhanced visualizations by improving multi-line text rendering with color-coded backgrounds for better readability. πŸ–ΌοΈβœ¨

πŸ“Š Key Changes

🎯 Purpose & Impact

UltralyticsAssistant commented 1 week ago

πŸ‘‹ Hello @RizwanMunawar, thank you for submitting an ultralytics/ultralytics πŸš€ PR! Your contribution to enhancing visualization in classification plots is greatly appreciated. 🎨 To ensure a smooth process, please review the following checklist:

πŸ“Š For further guidance, please refer to our Contributing Guide. If you encounter any issues or have additional questions, feel free to comment here, and we'll assist you.

🌟 Your proposed updates to text rendering and plotting aesthetics look promising. Dynamic text coloring and background rectangles will undoubtedly improve the visualization and user experience. Thank you for including a well-documented diff and comparative media to showcase your work. πŸ–ŒοΈβœ¨

This is an automated response to help streamline the process, but rest assured that an Ultralytics engineer will review your PR in detail and provide further guidance soon. πŸš€ Thank you for contributing to Ultralytics!

codecov[bot] commented 1 week ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 72.75%. Comparing base (c1860b8) to head (ef20fa1).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #18978 +/- ## ========================================== - Coverage 73.05% 72.75% -0.31% ========================================== Files 129 129 Lines 17486 17484 -2 ========================================== - Hits 12775 12721 -54 - Misses 4711 4763 +52 ``` | [Flag](https://app.codecov.io/gh/ultralytics/ultralytics/pull/18978/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ultralytics) | Coverage Ξ” | | |---|---|---| | [Benchmarks](https://app.codecov.io/gh/ultralytics/ultralytics/pull/18978/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ultralytics) | `34.07% <0.00%> (+<0.01%)` | :arrow_up: | | [GPU](https://app.codecov.io/gh/ultralytics/ultralytics/pull/18978/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ultralytics) | `37.70% <44.44%> (-0.24%)` | :arrow_down: | | [Tests](https://app.codecov.io/gh/ultralytics/ultralytics/pull/18978/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ultralytics) | `66.60% <100.00%> (-0.31%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ultralytics#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

sentry-io[bot] commented 1 week ago

πŸ” Existing Issues For Review

Your pull request is modifying functions with the following pre-existing issues:

πŸ“„ File: ultralytics/engine/results.py

Function Unhandled Issue
plot error: OpenCV(4.10.0) :-1: error: (-5:Bad argument) in function 'polylines' ultralytics....
Event Count: 1
plot TypeError: Argument 'thickness' is required to be an integer ultralytics.utils.plotting ...
Event Count: 1

Did you find this useful? React with a πŸ‘ or πŸ‘Ž

Laughing-q commented 5 days ago

@RizwanMunawar yeah nice idea! Personally I do have issues of current style of classify plotting when the background is white(basically can not see the plotted names). I think you could start this PR with the box_style argument we already have for Annotator.text method: https://github.com/ultralytics/ultralytics/blob/c5ac5548aeac8ed1554ef6224bd3d839974abc5b/ultralytics/utils/plotting.py#L494-L494 which I created this style(similar with the one you proposed in this PR) for visualizations when training/validating classification task. A val_batch0_pred.jpg example below: val_batch0_pred Ideally if we want this style we could implement this by directly adding box_style=True here: https://github.com/ultralytics/ultralytics/blob/c5ac5548aeac8ed1554ef6224bd3d839974abc5b/ultralytics/engine/results.py#L565-L565 and pass different color based on the predicted categories, instead of (255, 255, 255).

RizwanMunawar commented 4 days ago

@Laughing-q Wow, I didn't realize the same code had been used for plotting during the training. Your idea looks good, Let me test it.

Edit: BTW, I just performed testing and I have noticed the changes we made are not used in training and validation, these are only related to predictions i.e yolo classify predict. Additionally If I set box_style=True, annotator.text([x, x], text, txt_color=(255, 255, 255), box_style=True). I am getting the mentioned output.

bus

I think the main issue here is, that text includes not a single class, it's a combination of 5 classes, which I believe can not be plotted directly so loop will be required.

Laughing-q commented 4 days ago

@RizwanMunawar I've updated the code a bit to reuse the box_style argument, currently we just add box_style=True in Results.plot() for classification then it's all good.

RizwanMunawar commented 4 days ago

@Laughing-q ok, I will test it sometime today. In the meantime can you please attach some output results with new changes you made?

Laughing-q commented 4 days ago

@RizwanMunawar here it is! with box_style=True: bus box_style=False: bus

Also the training/validation plots become using the same red color, but I think it's ok since the category context is still different. val_batch0_labels

RizwanMunawar commented 4 days ago

@Laughing-q ok, Thanks. so this PR is ready?

Laughing-q commented 4 days ago

good to go from my side

RizwanMunawar commented 3 days ago

@Laughing-q ok thanks, @glenn-jocher Hi, can you please take a look, If everything is good, maybe we can merge this PR. Thanks